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1. Introduction 

In this paper we consider the eigenvalue problem, intersection theory of homogenous 
spaces (in particular, the Horn problem) and the saturation problem for the symplectic 
and odd orthogonal groups. The classical embeddings of these groups in the special 
linear groups will play an important role. We deduce properties for these classical groups 
from the known properties for the special linear groups. The tangent space techniques 
from [Bi] play a crucial role. Another crucial ingredient is the relationship between the 
intersection theory of the homogeneous spaces for Sp(2n) and S0(2n + 1). 

Let G be a semisimple, connected, complex algebraic group with a maximal compact 
subgroup K. Let f)+ be the positive Weyl chamber of G. Then, there is a bijection 
C : f)+ — > t/fC, x I— >■ ix, where t is the Lie algebra of K acts on t by the adjoint 
representation and ix denotes the i^-conjugacy class of ix. Define the eigencone (for any 
s > 1): 

s 

r(s, K) = {{ki, ...,ks)e {i/Ky : 3kj G kj with ^ kj = 0}. 

i=i 

For an algebraic group homomorphism G G' which takes K K', there is an 
induced map T{s,K) — > T{s,K'). Therefore, T{s,K) is functorial. In the case of 
Sp(2n) ^ SL(2ri) (similarly S0(2n + 1) SL{2n + 1)), we have stronger properties 
described below: 

Let p)^p(2") be the Cartan subalgebra for Sp(2ri) and similarly for S0{2n + 1), SL(2n) 
and SL(2n + 1). There are natural (linear) embeddings (see Sections 2.2 and 2.3): 

(^Sp(2n) ^ ^SL(2n)^ ^SO(2„+l) ^ j^SL(2„+l)_ 

Identify t/K with [)^ under C as above. We state our first main theorem (cf. Theo- 
rem 33): 

Theorem 1. (a) For hi, . . . ,hs G 

(/ii, ...,hs)e T{s, Sp(2n)) ^ {h, ...,hs)e T{s, SU{2n)). 

1 
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(b) Forh„...,hsei)f^'-'^'\ 

(hi,...,hs)e T{s, S0(2n + 1)) ^ {hi, ...,hs)e T{s, SU{2n + 1)). 

Our proof of this theorem is a consequence of a surprising result in intersection theory 
described below. 

1.1. Intersection theory. Let V be a 2n-dimcnsional complex vector space equipped 
with a nondegenerate symplectic form ( , ). Recall that if A = {ai < 02 < ■ ■ ■ < a^} is 
a subset of [2n] := {1, . . . , 2n} of cardinality m and F, a complete flag on C^", then, by 
definition, the corresponding Schubert cell Qa{F.) := {X G Gr(m, C^"^) : dimXriF^ = £ 
for a£ < u < ae^i,i = l,...,m}, where Gr(m, C^") is the ordinary Grassmannian of 
m-dimensional subspaces of C^". 

Let A^, . . . be subsets of [2n] each of cardinality m. Let E], . . . be complete 
isotropic flags on V in general position. The following second main theorem (cf. Theo- 
rem 10) is a key technical result that underlies the proof of the first main theorem and, 
in fact, is central to our paper. 

Theorem 2. The intersection n*^]^f2^j(i?f ) of subvarieties of Gr{m,V) is proper (pos- 
sibly empty) for complete isotropic flags El on V in general position. 

We have the same result for S0(2n + 1) (cf. Theorem 30). Let V be a vector space of 
dimension 2n-\-l equipped with a nondegenerate symmetric bilinear form. Let A^, . . . ,A^ 
be subsets of [2n + 1] each of cardinality m. Let E],. . . ,El be isotropic flags on V in 
general position. 

Theorem 3. The intersection Hj^i^lAiiEi) of subvarieties o/Gr(m, F') is proper. 

We remark that the above intersection result is false for S0(2n), since Corollary 11 
can easily seen to be violated in this case even for m = 1. 

1.2. The saturation problem. Any dominant weight A of SL(2n) restricts to a dom- 
inant weight Ac of the symplectic group Sp(2n). Similarly, any dominant weight A of 
SL(2n + 1) restricts to a dominant weight A^ of the orthogonal group S0(2n + 1). The 
following theorem (cf. Theorem 41) is proved geometrically by the method of "thcta sec- 
tions" . We obtain invariants in tensor products by constructing divisors in the products 
of isotropic flag varieties ( "the theta divisor" ) . This technique originates in our context 
in [B2]. In addition to methods from [B2], [Bi], we use some ideas of Schofield [S] in a 
crucial manner. 

Theorem 4. Let Vai,...,Vas be irreducible representations of SL{2n) (with highest 
weights X^,...,X'^ respectively) such that their tensor product has a nonzero SL(2n)- 
invariant. Then, the tensor product of the representations o/Sp(2n) with highest weights 
A^, . . . , A^ has a nonzero Sp{2n) -invariant, where A;^ is the restriction of to the Car- 
tan subalgebra o/Sp(2n). 

A similar property holds for the odd orthogonal group S0(2n -|- 1). 
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Theorem 5. Let Vxi, . . . , V\s be irreducible representations o/SL(2n+ 1) such that their 
tensor product has a nonzero SL(2n + l)-invariant. Then, the tensor product of the 
representations ofSO{2n-\-l) with highest weights A]^, . . . , has a nonzero S0(2ra + 1)- 
invariant, where A-^ is the restriction of to the Cartan subalgebra of S0{2n + 1). 

Using the saturation theorem of Knutson-Tao [KT], together with our Theorems 4 
and 1, we obtain the following improvement of the general Kapovich-Millson saturation 
theorem in the case of Sp(2n) (cf. Theorem 43). 

Theorem 6. Given dominant integral weights /i^,...,/!^ of Sp{2n), the following are 
equivalent: 

(1) For some N > 1, the tensor product of representations o/ Sp(2n) with highest 
weights Nfi^, . . . , N/j,^ has a nonzero Sp{2n) -invariant. 

(2) The tensor product of representations with highest weights 2/1^, ... ,2/1^ has a 
nonzero Sp{2n) -invariant. 

By carrying out a similar analysis for the odd orthogonal groups, we obtain the fol- 
lowing (cf. Theorem 44). 

Theorem 7. Given dominant integral weights i/^, . . . ,1/^ of S0(2n + 1), the following 
are equivalent: 

(1) For some N > 1, the tensor product of representations with highest weights 
Niy^, . . . , Nu^ has a nonzero S0(2n + 1) -invariant. 

(2) The tensor product of representations with highest weights 21/^ , ... ,21/^ has a 
nonzero S0{2n + 1) -invariant. 

1.3. Horn's problem for symplectic and odd orthogonal groups. There are two 
senses in which the term "Horn's problem" is generally applied: 

(A) Determination of a system of inequalities for the eigencone r{s,K), which is 
"cohomology free" (but perhaps recursive). 

(B) A system of inequalities characterizing nonvanishing structure coefficients in the 
cohomology H*{G/P) with the standard cup product and also {H* {G / P) , Qq) , 
where recall that ©0 is a deformation of the cup product in the cohomology of the 
flag variety G/P introduced in [BK]. In this paper, we only consider the ques- 
tion in the case of {H* {G / P) , Qo) ■ The system of inequalities can reasonably 
be expected to be parameterized by the cohomology {H*{L/Q), 0o), where L is 
a Levi subgroup of G and Q C L is a parabolic subgroup. It was suggested in 
[BK] that nonvanishing structure coefficients in the deformed cohomology can be 
characterized recursively. Recently, Richmond [R] showed that the cohomology 
(if*(S'L„/P), ©o) of partial flag varieties is strongly recursive (not just nonvan- 
ishing of structure coefficients, but also the actual structure coefficients). 

We consider (A) and (B) for the groups G = Sp(2n) and S0(2n -|- 1). Theorems 4 and 5 
imply (A), because the eigenvalue problem for the special linear groups is recursive by 
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Horn's original conjecture, proved by the combined works of Klyachko and Knutson-Tao 
(see [F4] for a survey) . 

It follows from [BK] that (B) implies (A). Wc reduce the Horn problem (B) for 
{H*{G/P),Qq) for the groups G = Sp(2n) and S0(2n + 1) and maximal parabolic 
subgroups P to the corresponding problem for the Lagrangian Grassmannians in Sec- 
tions 8 and 9 (cf. Theorems 48 and 51). Now, the Horn problem for the Lagrangian 
Grassmannians was solved by Purbhoo-Sottile [PS]. It should be noted that the solution 
of (A) obtained through the solution for (B) is different from the one obtained from 
Theorems 4 and 5. 

We make a conjecture (cf. Conjecture 47) generalizing many of the results in the paper 
for a diagram automorphism of G. 

We establish the basic notation in Section 2 and use them through the paper often 
without further explanation. 

Acknowledgements. Both the authors were supported by the FRG grant no DMS-0554247 
from NSF. 

2. Notation and Preliminaries 

Let G be a connected scmisimple complex algebraic group. We fix a Borel subgroup 
B and a maximal torus H C B. Their Lie algebras are denoted by the corresponding 
Gothic characters: Q, b, 1^ respectively. Let W — N{H) /H be the Weyl group of G, N{H) 
being the normalizer of H in G. Let C f|* be the set of positive roots (i.e., the set of 
roots of b) and A = {ai, . . . , a„} C the set of simple roots. Let {a^, . . . , a)^} C f) be 
the set of corresponding simple coroots and {si, . . . , s„} C be the set of corresponding 
simple reflections. Let f)+ C F) be the dominant chamber defined by 

[)+ = {x G f) : ai{x) G M+Vai}, 

where 1R+ is the set of nonnegative real numbers. For any 1 < i < n, let a;^ G ()* denote 
the i-ih. fundamental weight defined by 

uji{a)) = Si J. 

Let i? C P be a (standard) parabolic subgroup with the unique Levi subgroup L 
containing H. We denote by the set of minimal-length coset representatives in the 
Wp-cosets W/Wp, where Wp is the Weyl group of P (which is, by definition, the Weyl 
group of the Levi subgroup L). For any w G W^, we have the Bruhat cell 

A^ := BwP/P C G/P. 

This is a locally closed subset of the fiag variety G/P isomorphic to the affine space 
A^^'^\i{w) being the length of w. Its closure is denoted by A^, which is an irreducible 
projective variety (of dimension i{w)). We denote by 

G //2(dimc(G/P)-^(«,)) (^/p) 
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the cycle class of the subvariety A^, where H*{G/ P) is the singular cohomology of G/ P 
with integral coefficients. Then, by the Bruhat decomposition, {[A^]}^„gp^AP is an integral 
basis of H*{G/P). Define the basis {xi, . . . , x„} of i) dual to the basis {ai, . . . , of 
f)*, i.e.. 

Let X{H)^ be the set of dominant characters of H. For any A G X{H)^, let Vx be the 
finite dimensional irreducible G-module with highest weight A. This sets up a bijective 
correspondence between X{H)^ and the set of isomorphism classes of finite dimensional 
irreducible G-modules. Taking the derivative, we get an embedding X{H)^ D, where 

D :^ {A e r : A(q;,^) eK+Va,^} 

is the set of dominant weights. If G is simply-connected, X{H)+ can be identified with 

:= {Aer :A(a,^)eZ+Van, 
where Z+ is the set of nonnegative integers. 

We now give more specific details about the groups SL(n + 1), Sp(2n) and S0(2n + 1) 
below (see, e.g., [BL, Chapter 3]), since they will be of special interest to us in the paper. 

2.1. Special Linear Group SL(n + 1). In this case we take B to be the (standard) 
Borel subgroup consisting of upper triangular matrices of determinant 1 and H to be 
the subgroup consisting of diagonal matrices (of determinant 1). Then, 

^ = {t = diag(ti, . . .,tn+i) ■^ti = 0}, 

and 

i)+ = {t e i) : ti e Mandti > • • ■ > tn+i}- 

For any I < i < n, 

ai(t) =ti- U+i, a{ = diag(0, . . . , 0, 1, -1, 0, . . . ,0);a;i(t) = h -\ U, 

where 1 is placed in the i-th place. 

The Weyl group W can be identified with the symmetric group -S'^+i which acts via the 
permutation of the coordinates of t. Let {ri, . . . ,r„} C Sn+i be the (simple) refiections 
corresponding to the simple roots {ai, . . . , a„} respectively. Then, 

n = (i,i + 1). 

For any 1 < m < n, let P^ D B he the (standard) maximal parabolic subgroup of 
SL(n + 1) such that its unique Levi subgroup Lm containing H has for its simple roots 
{ai, . . . , am, ■ ■ ■ , ctn}- Then, SL(n + 1)/Pm can be identified with the Grassmannian 
Gr(m, n + 1) = Gr(m, C""^-*^) of m-dimensional subspaces of C"'*'^. Moreover, the set of 
minimal coset representatives W^"" of W/Wp^ can be identified with the set of m-tuples 

S{m, n + 1) = {A := 1 < ai < • • • < < n + 1}. 

Any such m-tuple A represents the permutation 

Va — (Oi, . . . , a^, ttjn+l, ■ ■ ■ , On+l)) 
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where {a^+i < ■ ■ ■ < a„+i} = [n + 1] \ {ai, . . . , a„,} and 

[n + 1] :={!,. ..,n+l}. 

For a complete flag E. : = Eq C C ■ ■ ■ G E^+i = C"+S and A e S{m,n + 1), 
define the corresponding shifted Schubert cell inside Gr(m, n + 1): 

^a{E,) = {M e Gr(m, n+1) : for anyO < i < mand anya^ < 6 < a^+i, dimMnE'fe = £}, 

wliere we set aQ = and flm+i = n + 1. Tlien, Qj^{E,) = g{E,)Af^'^, where g{E,) is an 
element of SL(n + 1) which takes the standard flag E° to the flag E,. (Observe that 
g{E,) is determined up to the right multiplication by an element of B.) Its closure in 
Gr(m, n + 1) is denoted by Q,a{E,) and its cycle class in i7*(Gr(m, n + 1)) by ^a\. 
(Observe that the cohomology class [fi^] does not depend upon the choice of E,.) For 
the standard flag E, — E°, we thus have flA{E,) — Afj. 

Remark 8. Note that, in the literature, it is more common to denote the Schubert cell 
by Q°^{E.) and its closure by f2^(i?.). For notational uniformity we have denoted the 
Schubert cell by VLa{E,), and its closure by VLa{E.). 

2.2. Symplectic Group Sp(2n). Let V = C^" be equipped with the nondegenerate 
symplectic form ( , ) so that its matrix ((cj, Cj)) .^^^ in the standard basis {ei, . . . , e2n} 
is given by 

J 
-J 



E 



where J is the anti-diagonal matrix (1, . . . , 1) of size n. Let 

Sp(2n) := {g e SL(2n) : leaves the form ( , ) invariant} 

be the associated symplectic group. Clearly, Sp(2n) can be realized as the fixed point 
subgroup G"^ under the involution cr : G — > G defined by (t{A) — E{A*)~^E'^ , where 
G = SL(2n). The involution a keeps both of B and H stable, where B and H are as 
in the SL(2n) case. Moreover, B'^ (respectively, H'^) is a Borel subgroup (respectively, 
a maximal torus) of Sp(2n). We denote B^^H" by B^ = B'^",!!'^ = H'^" respectively 
and (when confusion is hkely) B,H by _B^2n-i ^ ^A2„-i respectively (for SL(2n)). Then, 
the Lie algebra of H'^ (the Cartan subalgebra 1^^) 

f)^ = {diag(ii, ...,tn, -tn, -ti) : ti G C}. 

Let A'^ = . . . , Pn} be the set of simple roots. Then, for any 1 < i < n, /3i = a^jc, 
where {ai, . . . , a2n-i} are the simple roots of SL(2n). The corresponding (simple) coroots 
{Pi,---,l3n} are given by 

Pi = oii + «2n-i> for 1 < i < n 

and 
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Thus, 

[)^ = {diag(ti, . . . ,tn, —tn, - I —ti) : each ti is real andti > ■ ■ ■ > tn > 0} . 
Moreover, f)^^""^ is cr-stable and 

Let be the (simple) reflections in the Weyl group W'~^ = W^" of Sp(2n) 

corresponding to the simple roots . . . , respectively. Since H^^"-^ is o"-stable, 
there is an induced action of a on the Weyl group S2n of SL(2n). The Weyl group W'-^ 
can be identified with the subgroup of S2n consisting of cr-invariants: 

{(ai, . . . ,a2„) e S2n ■ a2n+i-i = 2n + 1 - OiVl < i < 2n}. 

In particular, w = (ai, . . . , a2n) G W-^ is determined from (ai, . . . , a„). 
Under the inclusion C 5'2„, we have 

Si = rir2n-i, if 1 < i < n - 1 

(1) = r„, if i = n. 

Moreover, for any u,v & W^' such that l'~^{uv) = £'"{u) + i'"{v), we have 

(2) (mt;) = £^^"-1 (u) + t^-' {v), 

where (w) denotes the length of w as an element of the Weyl group IV" of Sp(2n) and 
similarly for £"^2n-i _ 

For 1 < r < n, we let IG(r, 2n) = lG{r,V) to be the set of r-dimensional isotropic 
subspaces of V with respect to the form ( , ), i.e., 

IG(r, 2n) := {M e Gr(r, 2n) : {v, v') =0, Vt;, t;' e M}. 

Then, it is the quotient Sp(2n)/P^^ of Sp(2n) by the standard maximal parabolic sub- 
group with A*^ \ as the set of simple roots of its Levi component L^. (Again 
we take to be the unique Levi subgroTip of P^" containing H'~^ .) It can be easily seen 
that the set of minimal-length coset representatives of /Wpc is identified with 
the set 

6(r, 2n) = {/ := 1 < ii < • • • < < 2n and 7 n / = 0}, 

where 

(3) /:={2n+l-ii,...,2n + l-v}. 

Any such I represents the permutation wi — (ii, . . . , i„) e W'~' by taking {i^+i < • • • < 
U=[n]\(7u/). 

Definition 9. A complete flag 

E.:O^EoCEiC...CE2n = V 

is called an isotropic fiag if = E2n-a, for a = 1, . . . ,2n. (In particular, En is a 
maximal isotropic subspace ofV.) 
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For an isotropic flag E, as above, there exists an element k{E,) G Sp(2n) which takes 
the standard flag E° to the flag E,. (Observe that k{E,) is determined up to the right 
multiplication by an element of .) 

For any / e &{r,2n) and any isotropic flag E,, we have the corresponding shifted 
Schubert cell inside IG(r, V): 

^i{E,) = {M e IG(r, V) : for any < ^ < r and any ii < a < i^+i, dim MnEa^i}, 

where we set io — and ir+i — 2n. Clearly, set theoretically, 

(4) ^i(E,)^nj{E.)nlG{r,Vy, 

this is also a scheme theoretic equality (cf. Proposition 16 (4)). Moreover, $/(£'.) = 

pC 

k{E,)Kwi . Denote the closure of $/(£',) inside IG(r, V) by and its cycle class in 

if*(IG(r, V)) (which does not depend upon the choice of the isotropic flag E,) by [!»/]. 

For the standard flag E, = E°, we have = A^f . 

2.3. Special Orthogonal Group S0(2n + 1). Let V = C^^^^ be equipped with the 
nondegenerate symmetric form ( , ) so that its matrix E — ((^i, ej))^^^^.^2„+i (i^ the 
standard basis {ei, . . . , 62^+1}) is the (2n + 1) x (2n + 1) antidiagonal matrix with I's all 
along the antidiagonal except at the (n + l,n + l)-th place where the entry is 2. Note 
that the associated quadratic form on V is given by 

n 

tit2n+2-i- 

i=l 

Let 

S0(2n + 1) := {g e SL(2n + 1) : leaves the quadratic form Q invariant} 

be the associated special orthogonal group. Clearly, S0(2n + 1) can be realized as the 
fixed point subgroup under the involution 6 : G ^ G defined by 6[A) = E~^ {A^)~^ E , 
where G = SL(2n + 1). The involution 9 keeps both of B and H stable. Moreover, B^ 
(respectively, H^) is a Borel subgroup (respectively, a maximal torus) of S0(2n + 1). We 
denote B^, by B^ = S^", = if-^" respectively. Then, the Lie algebra of (the 
Cartan subalgebra f)^) 

f)^ = {diag(ii, ...,tn,0, -tn, -ti) : ti e C}. 

This allows us to identify with f)^ under the map 

diag(ti, . . . , t„, -t„, . . . , -ti) ^ diag(ti, . . . , t„, 0, -t„, . . . , -ti). 

Let A-^ = . . . , 5n} be the set of simple roots. Then, for any 1 < i < n, 5i = ai\^^B, 
where {ai, . . . , Q;2n} are the simple roots of SL(2n + 1). The corresponding (simple) 
coroots {51, . . . , (5^} are given by 

^'i = (4 + «2n+i-i> for 1 < i < n 
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and 

Thus, under the above identification, 

= f)5- 

Moreover, 1^^^" is ^-stable and 

Let {s'l, . . . , s^} be the (simple) reflections in the Weyl group = 1^^" of S0(2n + 1) 

corresponding to the simple roots {5i, .... respectively. Since iJ^^n jg ^-stable, there 
is an induced action of 9 on the Weyl group S2n+i of SL(2n + 1). The Weyl group 
can be identified with the subgroup of S2n+i consisting of ^^-invariants: 

{(Oi, . . . , a2n+l) e S2n+1 ■ a2n+2-j = 2n + 2 - VI < i < 2n + 1}. 

In particular, w = (ai, . . . , a2n+i) ^ is determined from ( ). (Observe 

that ttn+i = n+1.) We can identify the Weyl groups W'-'' ~ under the map 
(oi, . . . , a2n) ^ (oi, . . . , a„, n + 1, a„+i + 1, . . . , + 1)- 
Under the inclusion C 5'2n+i, we have 

s'i = rir2n+i-i, if 1 < ^ < n - 1 
(5) = r„r„+ir„, if i = n. 

For 1 < r < n, we let OG(r, 2n + 1) = OG(r, y) to be the set of r-dimensional 
isotropic subspaces of V with respect to the quadratic form Q, i.e., 

OG(r, 2n + 1) := {M G Gr(r, V') : Q(t;) = 0, Vt; G M}. 

Then, it is the quotient S0(2n+ 1) / P,^ of S0(2n + 1) by the standard maximal parabolic 
subgroup with \ {Sr} as the set of simple roots of its Levi component . (Again 
we take Lf to be the unique Levi subgroup of Pf containing H^.) It can be easily seen 
that the set of minimal-length coset representatives of /Wps is identified with 
the set 

6'(r, 2n + l) ^ {J < ji < ■■ ■ < jr <2n + l,jpy^n + l for any p and J n J' = 0}, 
where 

J':= {2n + 2-ji,...,2n + 2-jV}. 
Any such J represents the permutation w'j = (ji, . . . ,jn) ^ by taking {jr+i < ■ ■ ■ < 

i„} = M\(juj'). 

Similar to the Definition 9 of isotropic fiags on V, we have the notion of isotropic fiags 
on v. Then, for an isotropic fiag E'^, there exists an element k{E'^) G S0(2n + 1) which 
takes the standard flag E'1 to the flag E'^. (Observe that k{E'^) is determined up to the 
right multiplication by an element of .) 
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For any J G 6'(r, 2n+ 1) and any isotropic flag E'^, we have the corresponding shifted 
Schubert cell inside OG(r, V): 

^j{E',) = {M e OG(r, V) : for any < £ < r and any <a< j^+i, dim MnE'^^ £}, 
where we set jo = and jr+i = 2n + 1. Clearly, set theoretically, 

(6) ^j{E:) = nj{E:)nOG{r,V'y, 

this is also a scheme theoretic equality. Moreover, ^j(E'J = k{E[)K^^, . Denote the 

closure of inside OG(r, V) by ^ j{E',) and its cycle class in iJ*(OG(r, V")) (which 

does not depend upon the choice of the isotropic flag E'^) by j]- For the standard flag 
= E:, we have ^ j{E',) = A^f . 

3. Isotropic flags and proper intersection of Schubert cells in Gr(m, V) 

Fix a positive integer s. Let V = C"^"" be equipped with the nondegenerate symplectic 
form ( , ) as in Section 2, and let 1 < m < n be a positive integer. Let A^,. . . e 
S{m, 2n). The following theorem is a key technical result that underlies the proof of our 
theorem on the comparison of eigencone for Sp(2n) with that of SL(2n). 

Theorem 10. Let El,...,El be isotropic flags on V in general position. Then, the 
intersection of subvarieties r\j=i^Ai{Ei) inside GY{m,V) is proper (possibly empty). 

As an immediate consequence of the above theorem, we get the following: 

Corolleiry 11. Let 1 < m < n and let , . . . , P e 6(m, 2n) be such that 

s 

Yli^ij] e H*{IG{m,2n)). 

i=i 

Then, YVj^iinu] 7^ e i/*(Gr(m, 2n)). 

Proof. Observe that by [F2, Proposition 7.1 and Section 12.2] and [Bi, Proposition 1.1], 

s 

(7) Y[[^iA 7^ if and only if nj^^ $/. (^^) ^ 

for isotropic flags {Ei} such that the above intersection is proper. Thus, by assumption, 
$/j (-E'f) 7^ for such flags {Ei}. By the above theorem and Equation (4), we 
conclude that n^^iQp (£'f) 0. Prom this and using Equation (7) for Gr(m, F), the 
corollary follows. □ 

Before we can prove the above theorem, we need the following preliminary work. 

Given subsets / and K of [2n], we denote by |/ > K\ the number of pairs (i, k) with 
i E I, k E K and i > k. (We set \A > $\ — 0.) For K a singleton {k}, we abbreviate 
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|/ > K\ by \I > k\. Let us take an integer 1 < r < n. For an / G 6(r, 2n), we define 

/ = [2n] \ (/U/). 

(See Equation (3) for the definition of /.) We also set (for any I e 6(r, 2n)) 

sym2(/) = l(|/>/|+M^/;,)), 

and 

a2(/) = |/ > 7| -sym2(/), 

where fi{wj) represents the number of times the simple refiection s„ appears in any 
reduced decomposition of wj. 

With this notation, we have the following proposition. 

Proposition 12. For any I G &{r, 2n) and any isotropic flag E,, 

dim $/(£;,) = |/ > /| +sym^(/). 

Proof. Under the canonical inclusion W'-^ ^ 5'2„, 

yjj^-^u)i= (ii, . . . , . . . ,jn-r,'2n+l-jn-r, ■ ■ • , 2n+ 1 - ji , 2n+ 1 - V , . . . ,2n+l-ii), 
where {ji < ■ ■ ■ < jn-r} ■= [n] \ (/ U /). Now, by Equations (1), (2) and [Fi, §10.2], 

= i(|/>/| + |/>7| + |/>7|+Mw7)) 

= ^(|7 > /| + |7 > /| + |7 > i\+ii{wi)) 
= |/ > /| +sym^(/). 

This proves the proposition. □ 

3.1. Tangent space of isotropic Grassmannians. We calculate now the tangent 
space T{X)m to X = IG(r, 1/) at a point M. Because of the natural embedding 
IG(r, V) C Gr(r, V), we have T{X)m C TGr(r, V)m = Hom(M, V/M). 

Clearly, M-*- is a 2n — r dimensional subspace of V that contains M and there is a 
canonical isomorphism V/M-^ ~ M* (induced from the symplectic form). Hence, we 
have an exact sequence 

Hom(M, M^/M) Hom(M, V/M) ^ Hom(M, V/M^) = Hom(M, M*) 0, 

obtained from the inclusions M C M-*- C V. It is clear that M^/M is a 2n — 2r 
dimensional space that possesses a nondegenerate symplectic form. Let Pm C Sp(2n) 
be the stabilizer of M and sym^ M* the space of symmetric bilinear forms on M. 
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Lemma 13. T(X)m = ^^-'^(sym^ M*) and hence there is an exact sequence of Pm- 
modules. 

(8) ^ Hom(M, M^/M) ^ T{X)m ^ sym^ M* 0. 

Proof. Let ijj : M ^ V/M he a. linear map (viewed as a deformation of the trivial map). 
The deformed M (obtained from ■0) needs to be isotropic. So, up to the first order, we 
have 

{v + e'ijj(v),v' + e'ijj(v')) v,v' E M. 

Hence, 

{v,ip{v')) + {ip{v),v')^0, 

or that 

This gives us the symmetric bilinear form (l)[ip){v,v') = {v,ip{v')). Hence, T[X)m Q 
^-^(sym^M*). But, 

dim T(X)m = dim IG(r, V) = ^(4n-3r + l) = dim(/)-^(sym2 M*), 

and hence T{X)m = ^-^(sym^ M*). □ 

Let E, be an isotropic flag on V . This induces flags on M, and hence on jM} 

Lemma 14. For any isotropic flag E, on V, the induced flag on M-^/M is isotropic 
with respect to the nondegenerate symplectic form on M^/M. 

Proof. Consider an element of the induced flag 

Ea n M^/Ea n M = {E2n-a + M)^ / Ea H M. 

Its perpendicular (in M^/M) is therefore (E2„_a+M)nM^/M = E2n-a^M^ / E2n-a^M , 
which is again a member of the induced flag on jM. □ 

3.2. Some crucial inequalities derived from nonvanishing intersection product. 

For any I e 6(r, 2n), isotropic flag E, on V and M e ^/(E,), we now calculate the 
tangent space T{^i{E,))m Q T{Qj{E,))m (with the notation as in Section 2). 

Lemma 15. M-^ e nj^j{E.). 

Proof By definition, / U / = [2n] \ I. Now, dim(^„ n M^) = dim((E2n_a + M)^) = 
2n-{r+2n-a-dim{E2n-a^M)) = a-r+dim{E2n-a^M). Hence, E^nM-^ ^ E^-idM-^ 
if and only if E2n-a n M = E2n+i-a n M, i.e., 2n + l- a^I<^a^I. □ 

For any a e [r], let Xa := \ia > -^|, where / = {1 < ii < • • • < < 2n}. Let 
X = IG(r,T/). 



There is exactly one way of inducing a complete flag on M^/M from E,. The elements of the flag 
are {M^/M) n (K + M/M). This can be written as ((K + M) n M^)/M = Ean M^/Ea n M. 
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Proposition 16. (1) 

Hom(M, M^/M) n T{yti{E.))M = 

{7 e Hom(M,M^/M) : 7(^(M),) C E{M^/M)x^,ya G [r]}, 

where E{M), and E{M-^ /M). are the induced complete flags on M and M-^/M 
respectively with the changed labels . by the dimension. The dimension of this 
vector space is \I > I\. 

(2) Hom(M, M^/M) n T{^i{E,))m = Hom(M, M^/M) n T{ni{E,))M. 

(3) 

<f>{T{^j{E,))M) = (f>{T{ni{E.))M n T{X)m) 

= {7 e sym^M* : ^{E{M),, E{M)J = 0,Va e [r]}, 

where ta — \I > ia\- Moreover, the dimension of this vector space is sym^(/). 
(4) We have an equality of schemes ^i{E,) — D,i{E,) fl X. 

Proof. We first prove Part (1): Prom the known description oiT{Qj{E,))M as the space 
of maps 7 : M ^ V/M so that 7(M n E^) C (Eb + M) /M for all 6, we see that for 7 to 
also be in Hom(M, M-'-/Af). the condition is 

(9) 7(M n Eb) C + M) n M^)/M for any 6 e [2n]. 

By Lemma 15, Ei, fl M-^/Eb fl M is of dimension |6 > / U J| - |6 > 7| = |6 > /| for any 
b e [2n]. Putting 6 = ia, the condition given in Equation (9) can be rewritten as 

j{E{M)a) C E{M^/M)x^ya e [r]. 

The dimension of this vector space is clearly |7 > 7|. This proves Part (1). In particular, 

(10) 

dim(Hom(M,M^/M)nT(<l>/(E.))M) < dim(Hom(M, M^/M)nT(l]j(E.))M) = |/ > /| = |/ > / 

where the last equality follows since /fl/ = 0. We have (by the description of T(0/(£',))m 
given above) 

0(T($/(E.))m) C <j){T{nj{E.))MnT{X)M) 

C {7 e sym^ M* : -f{Eb D M, E^ n M) = 0, V6 G [2n]} 

^^^^ = {7 e sym^ M* : 7(Efe n M, E2„_6 n Af ) = 0, V6 G [2n]} 

= {7 G sym^M* : 7(£;(M)„, £;(M)tJ = 0,Va G [r]}. 

(In the above, we have used dim{E2n-ia,f^M) = ta- Furthermore, if EbDM = Eb+idM, 
then the condition j{Ebr\M, £^2n-6nM) = implies the condition j{Eb+ir\M, £^2n-6-in 
M) = 0.) 

Moreover, the last space has dimension sym^(/) by the following calculation. 

As above, express the vector space under consideration in the more symmetric form 

= {7 e sym^ M* : ^(E^ n M, E2n-b n M) = 0, V6 G [2n]}. 
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Form the "analogous" space 

^2 = {7 e A^M* : 'yiEb H M, E2n-b n M) = 0, V6 G [2n]}. 

Clearly, 

T/i © T/2 = {7 e (M M)* : -f{{Eb n M) ® {E2n-b n M)) = 0, V6 e [2n]}, 
which, in turn, can be written as 

{7 G Hom(M, M*) : 7(^6 n M) C ann(E2n-6 H M), V6 G [2n]}, 

where ann(£^2n-6 H M) is the annihilator of E2n-b H M in M*. The last space is of 
dimension |/ > /|. For this note that ann(£'2n-i„ nM) is of dimension r — \I < 2n — ia\ — 
\I > 2n - ia\ ^ \I > 2n + l - ia\ = \ia > I\, since 7 n / = 0. 

Choose a basis of M compatible with the filtration {Ehr\M}b^i2n]- Then, in this basis, 
the vector spaces Vi and V2 are subspaces of symmetric and skew-symmetric matrices 
respectively. The difference dim Vi — dim V2 is the number of diagonal terms allowed in 
Vi, i.e., 

dim\/i-dimK2 = \{a G [r] : Ei^DM ^ E2n-i^nM}\ = \{a G [r] : i„ > 2n-ia}\ = \I > n\. 
Therefore, we conclude 

dim Vi + dim V2 = \I > I\ 

dimVi — dimV2 = \I > n\. 

Using Equation (16), we obtain dimT4 = sym^ I and dimT/2 = A^/. This proves the last 
assertion in Part (3). 

Combining Equations (10) and (11), and using Lemma 13, we get 

(12) dim(T($,(S.))M) < sym2(/) + |/ > /|. 

But, by Proposition 12, the above inequality is an equality. Hence, all the inclusions and 
inequalities in Equations (10)- (11) arc equalities. This proves Parts (2)-(3). 

By the definition, set theoretically $/(£',) = Qi{E,) CiX. Moreover, by Parts (2)-(3), 
the tangent space T{^i{E,))m = T(Q/(£;,))m n T{X)m- This proves Part (4) and thus 
completes the proof of the proposition. □ 

Let V be as in the beginning of this section and let r < n. Fix M G IG(r, V) and 
/ G 6(r, 2n). Let U = Ui{M) be the set of isotropic flags E, on V such that M G $/(-£,). 
A flag E, E U induces a complete flag E,{M) on M, and a complete isotropic flag 
E,{M-^/M) on M-^/M (Lemma 14). The following lemma follows by choosing basis 
elements appropriately. 

Lemma 17. The map U — > F1(M) x IFl(M-'-/M) is a surjective fiber bundle with irre- 
ducible fibers, where F1(M) is the variety of all the complete flags on M andW\{M-^ / M) 
is the variety of all the isotropic flags on M^/M. 
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Now, let y be a 2r-dimensional vector space with a nondegenerate symplectic form 
containing M of dimension r as an isotropic subspace. Let I^,...,P e 6(r, 2r) be 
written os P — {v[ < ■ ■ ■ < ii}. Define, for any j e [s] and a & [r], — \P > 

Lemma 18. The following are equivalent: 

{(5) For some (and hence generic) complete flags F}, . . . , on M, the vector space 
(13) {7 e sym^ M* \ ^{Fl F^) ^ 0, a E [r]J e [s]} 

is of the expected dimension 

s 

r{r + l)/2 - ^cosym^(P), 

where cosym^(P) :— — sym^(P). 

Proof. (q;)=^ (P): Choose generic isotropic flags E],. . . on V. Because of the as- 
sumption (a), n|^]^$/j ) is nonempty and, by simultaneously translating each E{ by 
a single element of Sp(2r), we can assume that M belongs to this intersection. Let 
F},. . . , F/ be the induced flags on M. 

The vector space (13) is the tangent space to the scheme theoretic intersection n^^i^/j (Ei) 
at M (by Proposition 16 (3)). Again applying Proposition 16 (3), the transversality of 
the intersection implies that (/?) holds. 

Now assume (/5). Choose (generic) complete flags on M such that (/5) 

is satisfied. Now choose isotropic flags El,...,E^ on V such that M e r\j^i^ij{Ei) 
and the induced flags on M are F}, . . . , F/ respectively. This is possible by Lemma 17. 
Using {(3) and Proposition 16 (3), we see that n|^;^$p(Ff) is transverse at M. Therefore, 
using standard facts from intersection theory on a homogenous space, we see that (a) 
holds. □ 

Let {ei}i<i<n be the basis of f)*^, where q is the diagonal matrix 

diag(0,...,0,l,0,...,0,-l,0,...,0), 

where 1 is placed in the i-th slot and —1 in the (2n-|- 1 — i)-th slot. Recall from Section 2 

that we have idcntiflcd f)*^ with i)^ and denote either of them by f). Let {ej}i<j<„ be the 
dual basis (of [)*). Let p'" (respectively, p^) be half the sum of positive roots of Sp(2n) 
(respectively, S0(2n + 1)). Then, 

2p^ = (2n - l)ei + (2n - 3)e2 + • • • + Scn-i + e„, 

and 

2p^ = (2n)ei + (2n - 2)e2 + • • • + 4e„_i + 2e„. 

Thus, 

2(p*^ — p^) — e, where e := ei + • • • + e„. 



16 



PRAKASH BELKALE AND SHRAWAN KUMAR 



Let {xf , . . . , x^} be the basis of f)"" as in Section 2 and similarly for . It is easy to see 
that 

xf — xf — ei-\ h if 1 < i < n — 1 and 

(14) x^ = 2x^ = ei + --- + en. 



Lemma 19. For any / G 6(r, 2n), 

wi{er+i H h e^) = H h e^^,^, 

for some 1 < ji < j2 < ■ ■ ■ < jn-r < n. 

Proof. This follows easily from the definition of 6 (r, 2n) by observing that the action 
of C on — is given by the standard action of on ©j^j^ Ce^ and 

making the identification Cj — —e2n+i-j, for any 1 < j < 2n. □ 

Lemma 20. For any w e 1^^, 

we e ±ei ± 62 ± • • • ± e„. 

Moreover, 

(15) = # negative signs in the above = -^{n — {we, e)), 

where {, ) is the normalized W'-' -invariant form on i)* given by (e^, e^) = Sij. 
In particular, for any / e (5(r, 2n), 

(16) fJ'iwi) = |/ > n\. 

Proof. Since the symmetric group Sn C (acting as the permutation group of {ei, . . . , €„}) 
acts trivially on e, the first part follows easily. Applying s„ to ±ei ± + • • • ±e„, the num- 
ber of minus signs either increases by 1 or decreases by 1, whereas the application of any 
Si to ±ei ± H — - it e„ (for any ?' < n) keeps the number of minus signs unchanged. Thus, 
the number of negative signs in we < fi{w), for any w G W*" . Further, the last equality 
in Equation (15) is trivially satisfied. Now, the longest element Wq G W-^ makes all the 
signs in woe negative and ij,{wq) — n. Prom this Equation (15) follows for wq and hence 
for any w G VF*^. Otherwise, taking a reduced decomposition of w and extending to a 
reduced decomposition of Wq, we would get a contradiction. 

Equation (16) follows from Equation (15) immediately. □ 

Corollary 21. For any / G ©(r, 2n), 

{wj^e){x^)^r-2pi{wi). 

Proof. 

{wj^e){xf) = {wj^e, e) - {wj'^e, e^+i H h e„) 

— n — 2^{wj) — (e, wi{er+i + • — h e„)), by Equation (15) 

— n — 2^{wi) — {n — r), by Lemma 19 
= r - 2ii{wi). 
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Fix I^, . . . ,P e 6(r, 2n) and define functions 9^ ,9^ : 6(r, 2n) — >• Z by 
and 

wliere xS/ •= (P*" + '^T^P^) Xwi is defined similarly. 
Lemma 22. For r < n and any I e &{r, 2n), 

9^{i)^9^{i)+m-Y.n{n 

where p,{I) :— r — ii{wi). Similarly, for r — n, 

29^{i) = 9''{i)+m-Y.p{n 

Proof. Assume first that r < n. In this case 

9^{I) - 9^{I) = i((6 + wj'e) - E + <^))i^r) 

— r — n{wj) — ^^(r — fi{wij)), by Corollary 21 

Now, we consider the case r — n. In this case, 

29^{I) - 9^{I) = ((e + ^7^6) - X] (6 + w-^e)){x^^) 

i=i 

This proves the lemma. 

Lemma 23. Le^ I^,. . . ,P e S(r, 2r) &e suc/i t/ia^ 

s 

l[[^j,]^0eH*{lG{r,2r)). 
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Then, 

s s 

^ > r - (dimIG(r, 2r) - ^ codim($jj)). 

Observe that, by Equation (16), 

(17) Ji{P) = \r> P\. 

Proof. Let / G ©(r, 2r) be such that [$/] appears with nonzero coefficient in the product 
11^=1 [^/j]- The maximal parabohc subgroup is minuscule and hence by [BK, Lemma 
19], e^{I) = 0. Thus, by Lemma 22, 

(18) 9^{I)^-p{I)+j2Hn- 
But, clearly, 

s 

Ijl{wi) < dim $7 = dimIG(r, 2r) - ^ codim($p). 

Hence, 

s 

(19) = r — iJ,{wi) >r — dim IG(r, 2r) + codim($p). 
Also, by Theorem 55 and [BK, Proposition 17(a)], 

(20) e^{i) > 0. 

Combining Equations (18)- (20), wc get the lemma. 

An alternative proof of the lemma: The following is a quick (though less con- 
ceptual) way to prove the lemma. Let 1° — {r,r + 2, . . . , 2r}. Then, by Proposition 12, 
codim$7o — 1 and [$70] is ample on IG(r, 2r). So, by cupping with a sufficient number 
of [$70], we are reduced to the case dimIG(r, 2r) — J2j codim($7j) = 0. 

Suppose now by way of contradiction that "^^j^ifiiP) < r. Choose generic isotropic 
flags Ej,. . . ,El on a 2r-dimcnsional vector space V with a nondegenerate symplectic 
form, and let M e n^=i$7. (^;;). Let be the induced flags on M. Let t{ := \P > 
Since the intersection n*^^$7j(£'^) is transverse at M, by Proposition 16 (3), there are 
no nonzero symmetric bilinear forms on M, so that 

0(Fi,F;^.) = O, je[si ae[r]. 

Let M' := X]j=i which is a proper subspacc of M since, by assumption, Yl^j=i fii^) < 

r. Thus, sym^ (M/M')* ^ sym^M*. It is easy to see that any nonzero element (p 
in sym^(M/M')* satisfles the forbidden possibihty above (since at least one of a or 
ti < H^^) for any a e [r] and j e[s]). □ 
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Lemma 24. Let I^,. . . ,P G 6(r, 2n) he such that 

s 

J][$,,]^0e//*(IG(r,2n)). 
Then, the following inequalities hold: 

s 

(21) r(r + l)/2 - ^cosym2(P) > 0. 

s s 

(22) X]'"^^^) - ^ ~ (dimIG(r, 2r) - ^cosym2(P)), 
i=i j=i 

and 

s 

(23) r(r -l)/2-^ CO A^(/^) >0, 

where recall that sym^(7) and A^(/) are defined above Proposition 12, cosym^(7) is de- 
fined in Lemma 18, and 

coA^(7) :=r(r-l)/2- a2(J). 

Proof. Choose generic isotropic flags E],. . . ,El on V, and let M G nj^^^^/j (£^^). Let 
be the induced flags on M. Since the intersection nj^^$/j(£'f) is transverse at M, 
the images of the tangent spaces of ^j3{Ei) via the map : T(IG(r, 2n))M — ^ sym^ M* 
(deflned in Lemma 13) are transverse in sym^M*, as can be easily seen (see [PS] for a 
similar argument). This gives us Equation (21) in view of Proposition 16 (3). 

Let Pj be the order preserving bijection of P U P with [2r] and let 1^ be the subset 
(3{P) of [2r] (of cardinality r). Hence, we see using Lemma 18 and Proposition 16 that 

s 

l[[^,^]^0eH*ilGir,2r)). 

Observe next that \n > P\ — \r > I^\ and hence \P > n\ — \I^ > r\. Thus, by 
Equation (17), 

jl{P) = jl{P,) and sym\P) = sym2(/^). 

Now, we can use Lemma 23 and Proposition 12 to obtain Equation (22). Equation (23) 
follows from Equation (22) and the equality 

cosym2(P) - CO a2(P) = r - {sym\P) - A^{P)) = /i(P). 

□ 
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3.3. Isotropic flags and transversality. For any r < m < n define 
£(m, r, 2n) = {X e Gr(m, V) : dim(X n X^) = r}, 
where V is as in the beginning of Section 3. 

Lemma 25. (1) Sp(2n) acts transitively on C{m,r,2n) . 

(2) C{m, r, 2n) is a fiber bundle over IG(r, V) via the map tt defined by X M — 

xnx^. 

(3) For a fixed m, the union of the subvarieties jC{m, r, 2n) over all choices of < 

r < m is Gr(m, V). 

(4) The action o/Sp(2n) on Gr{m,V) has only finitely many orbits. 

Proof Let X, X' e C{m, r, 2n), M = X n X^ and M' = X' n X'-^. We need to produce 
g e Sp(2n) such that gX = X' . Since Sp(2n) acts transitively on IG(r, 2n), we may 
assume that M — M' . Let 

Gm ■■= {g G Sp(2n) : gM = M}. 

We easily see that Gm surjects onto the symplectic group of M-^/M by choosing sub- 
spaces N and Y ofV such that V ^ M®N®Y, dim N = dim M and (M ® N,Y) = 0. 
Now, 7r~^({M}) can be identified with the set of subspaces X C M-^/M of dimension 
m — r which are nondegenerate with respect to the induced (nondcgcncratc) symplectic 
form on /M. This reduces us to the case of r = (for the proof of the first part of 
the lemma). So, we need to show that Sp(2n) acts transitively on the set of subspaces 
N ol B. given dimension m such that N fl N-^ = (0). Let X be one such subspace and 
Y = X-^. Clearly, V = X (B Y and the induced symplectic forms are nondegenerate 
on X and Y. Choose "symplectic bases" {cj, /«} of X and {uj,Vj} of Y. Map them 
appropriately to the standard basis vectors in V. This transformation is in Sp(2n). This 
proves Part (1). Part (2) follows from the above description of 7r~^({M}). Part (3) of 
course is clear and Part (4) follows from Parts (1) and (3). □ 

For an isotropic flag E, on V, and subsets A e S{m, 2n) and / e 6(r, 2n), deflne the 
subvariety: 

^lAE,) ^{X e Gr(m, V) : X e VLa{E,),X n e ^i{E,)} C C{m, r, 2n). 

Lemma 26. Set L -.^ A\{IUI) and T := [2n] \ (/ U L U /). Assume that ^i^E,) is 
nonempty. Then, it is irreducible and 

dim^i^E,) = |/ > L| + |/ > T\+sym'^{I) + |L > T|. 

Moreover, I G A and I d A = 0. 

Proof. This follows from considering the fibration ttj^a '■ ^i,a{E,) — >■ ^i{E,), where tti^a 
is the restriction of tt to ^i^E,). (This is a surjective fibration because of the S^- 
equivariance.) By Proposition 12, the dimension of $/(£',) is 

|7 > L U T| + sym^(7) = |7 > L| + |7 > T| + sym^(I). 



EIGENCONE AND SATURATION 



21 



So, we just need to prove that the fiber dimension of ttj^a is |L > T|. 

Let M e ^i{E,), and X e Gr(m - r, M^/M) be such that the induced form on X 
is nondegenerate. Then, X := q^^[X) e ^j a{E,) if and only if X G Qa{E,), where 
q : M"*- — M-^/M is the quotient map. By definition, X G Qa{E,) if and only if 
dim(£;„ nX) = \a> A\ for all a. Now, by Lemma 15, the image of Ea n M-^ in M^/M 
is of dimension \a > [2n] \/| — |a>/| = |a>/| (see the footnote ^^''). Moreover, for 
any subspace X of M-*- containing M, 

dim(E„ n X) = dim(E„ n M) + dim((E„ + M/M) n X). 

Thus, for any X G Gr(m - r, M^/M),X g"-^(X) belongs to if and only if 

(24) dim(£;(M^/M)^(„) r\ X) ^ \a> A\ - \a> I\, Va G [2n], 

where (^(a) := \a> I\. 

We next claim that I <Z A. Write ^ = {l<ai<---<arn< 2n}. If possible, 
let a G / be such that Oj < a < Oj+i, for some < i < m (where we set Oq = and 
Om+i = 2n + 1). Then, 

|a > A| - |a > /| < |(a - 1) > A| - |(a - 1) > /|. 

By Equation (24), this gives 

dim(E(M^/M)^(,) nX) < dim(E(M^/M)^(,_i) nX), 

which is a contradiction. Hence the claim J C A is established. 

We further claim that IC^A = 0. If possible, take a G / such that a* := 2n + 1 — a G ^1. 
Then, 

\a* >A\- \a* >I\> \{a* - 1) > A\ - \{a* - 1) > /|. 
Thus, by Equation (24), 

dim(£;(M^/M)^(„*) n X) > dim(£;(M^/M)^(„*_i) n X). 

However, </?(a*) = (p{a* — 1), which is a contradiction. This proves that If] A — 0. Thus, 
for any X G Gr(m - r, M-^/M), X = q~^{X) belongs to ^i^a{E,) if and only if for any 
1 < p < 2n - 2r, 

(25) dim(E(M^/M)p n X) = \jp > L\, 

where / = {1 < ji < • • • < j2n-2r < 2n}. This shows that the fiber of ttj^a is an open 
subset of a Schubert cell in Gr(m — r, 2n — 2r) of dimension \L > T\ since T :— I\L. In 
particular, ^i^a{E,) is irreducible. This proves the lemma. □ 

Lemma 27. (1) The dimension of C{m,r, 2n) is dim(Gr(m, 2n)) — r(r — l)/2. 
(2) If nonempty, ^i^a{E,) is of dimension no greater than dim(Q^(£',)) — A^(/). 
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Proof. By Lemma 25, C{m,r,2n) is a fiber bundle over lG{r,V). Moreover, as in the 
proof of Lemma 25, the fiber is an open subset of Gr(m — r,2n — 2r). Therefore, 
dim jC{m, r, 2n) equals 

dimIG(r, V) + (m - r)(2n - 2r - (m - r)) = r(4n - 3r + l)/2 + (m - r)(2n - m - r) 

= dim(Gr(m, 2n)) - r(r - 1) /2. 

This proves the part (1) of the lemma. 
We next prove part (2). By Lemma 26, 

dim^i^AiE,) = |/ > L| + |7 > T] +sym2(7) + |L > r| 

= |7 > r| + |L > /| + |L > T| + |/ > /| - a\I) + \I > L\-\L> I\ 

(26) = dimQ^(E.) - a'^{I) + \I > L\ - \L > I\. 
The lemma will therefore follow from the inequality 

(27) \I>L\<\L>I\, 

which is proved below. 
We first show 

(28) \o, ^ L\ < \L > a*\ for any a G [2n], where a* := 2n + 1 — a. 

Take X G $/,a(^.) and let 7r(X) = M. Let F := X n D M n We look at the 
image oi X ^ Y* (induced by the symplectic form), where Y* is the dual of Y. The 
image has dimension 

dimF - dim(r n M) = \a > A\ - \a > I\ = \a > L\. 

But, clearly, {X D Ea*-i) + M goes to zero under the map X — > F* and hence the image 
of X — > y* is of dimension no greater than \L > a*\. This proves Equation (28). Hence, 

|/ > i^l < > /|. 

But since I H L = and also / fl L = 0, we get Equation (27) and hence Part (2) of the 
lemma is proved. □ 

3.4. Proof of Theorem 10. Choose isotropic fiags {E{}i<j<s such that the intersection 
n*^]^$/j (£'f ) is transverse and dense in n*^j^$/j (-Ef ) for all P G ©(r, 2n) and all 1 < r < 
m (cf. [BK, Proposition 3]). For any irreducible component C of r{j=i^Ai{Ei), there 
exists a dense open subset U ol C and an r together with subsets {I^}i<j<s C 6(r, 2n) 
such that U C £(m,r, 2n) and tt{U) C n^=i$7i(-Bf). 

It suffices to show that the dimension of any irreducible component of n^=i$/j,AJ (-E'f ) 
is no greater than dimGr(m, 2n) — X]j=i codim(n^3). 

Take (generic) isotropic fiags {Ei}i<j<s such that the intersection n^^^^p^^j (£^^) in 
>C(m, r, 2n) is proper for all the choices of P G S(r, 2n) and G S{m,2n). This is 
possible since Sp(2n) acts transitively on >C(m, r, 2n). 
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Therefore, using Lemma 27, any irreducible component of n|^^$p^^i(£'f) lias dimen- 
sion no greater than 



s 

dimGr(m,2n)-(r(r- l)/2) -^(|dimGr(m, 2n) - r(r - l)/2 - (dim Q^.- - a2(P))) 

s s 

= dimGr(m,2n) - (J^codimQ^,) - (^r(r - l)/2 - ^co a2(7^)). 
Using Equation (23), this last quantity is no greater than 

s 

dim Gr(m, 2n) — codim , 

as desired. This finishes the proof of the theorem. □ 

3.5. Analogue of Theorem 10 for S0(2n + 1). We now prove the analogue of The- 
orem 10 for S0(2n -|- 1). The proof is very similar and hence we will only give a brief 
sketch. First we need the following preparation. As in Subsection 2.3, V :— C^""^^ is 
equipped with the quadratic form Q. 
For any r < m < n define 

0{m, r, 2n+l)^{X e Gr(m, V) : dim{X n X^) = r}. 

We have the following analogue of Lemma 25 for S0(2n -|- 1). Its proof is similar and 
hence is omitted. 

Lemma 28. (1) S0(2n-|- 1) acts transitively on 0{'m,r,2n + 1). 

(2) 0{m,r,2n + 1) is a fiber bundle over 0G{r,2n + 1) via the map tt' defined by 

X ^ M = X nx^. 

(3) For a fixed m, the union of the subvarieties 0{m,r,2n + 1) over all choices of 
< r < m is clearly Gr(m, y). In particular, the action of S0{2n + 1) on 
Gr(m, V) has only finitely many orbits. 

For an isotropic flag E[ on V', and subsets A e S{m, 2n + 1) and J e 6'(r, 2n + 1) 
(cf. Subsection 2.3), define the subvariety of 0(m, r, 2n -|- 1): 

^jAK) = {Xe Gr{m,v') : X e nA{K),xnx^ e ^j{e:)}. 

Lemma 29. (1) The dimension of 0{m,r,2n+l) Z5 dim(Gr(m, 2n-|-l))— r(r + l)/2. 
(2) If nonempty, j^E',) is irreducible of dimension no greater than dim(r2^(ii^^)) — 
sym2(J). 

Proof. The fiber of the bundle tt : C(m, r, 2n -|- 1) — > OG(r, V) is an open subset of the 
set of choices of m-dimensional spaces containing a fixed r-dimensional subspace and 
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contained in the perpendicular space of this r- dimensional space. Therefore, by a simple 
calculation, dimO(m, r, 2n + 1) equals 

dim(OG(r, V')) + (m - r)(2n + 1 - 2r - (m - r)) 

= (r(r - l)/2 + r(2n + 1 - 2r)) + {m - r){2n + 1 - m - r) 

= dim(Gr(m, V')) - ^(^ + l)/2- 

This proves Part (1). 

We next observe that as in the symplectic case (Lemma 26), if nonempty, 

dim*j,^(E;) = \J > L'\ + \J >T'\ + A^{J) + \L' > T'\, 

where L' := A\{JU J'), T' := [2n + 1] \ ( J U L' U J') and J' is defined in Subsection 2.3. 
Similar to Equation (26), we have 

dim ^j,a(^0 = dim((]A(^I)) - sym=^(J) + |J > L'\ - \L' > J'\. 

The second part of the lemma will therefore follow from the following inequality whose 
proof is similar to that of Equation (27). 

(29) \J > L'\ < \L' > J'\. 

□ 

Theorem 30. Let A^, . . . be subsets of [2n+l] each of cardinality m. Let E'\, . . . , E'l 
be isotropic flags on V — C^'"'^^ in general position. Then, the intersection Dj^iQAiiE'i) 
of subvarieties of Gr(m, V') is proper. 

Proof. For any irreducible component C of ni^^ri^j (£"-^), there exists a dense open 
subset U' C C and an r together with subsets {J^}i<j<s C 6'(r, 2n+ 1) such that 
U' C e)(m,r,2n + 1) and 7r'(C/') C Cf^^^-^ jj{E'{). 

It suffices to show that the dimension of any irreducible component of j3,Ai{E'l) 

is no greater than dim(Gr(m, 2n-\-l))—Y^^-^-^ codim(QAj). The dimension of n^=i^j3^Aj(£"^) 
is no greater than 

dim(Gr(m, 2n + 1)) - r(r + 1) /2 

s 

- J^(|dim(Gr(m, 2n + 1)) - r(r + l)/2 - (dim(f}A,) - iijm^{P))^ 

s 

= dim(Gr(m, 2n + 1)) — r(r + l)/2 — ^^(codim(r2Aj) — cosym^(J-')). 

i=i 

Using Theorem 55 and Equation (21), this last quantity is no greater than dim(Gr(m, 2n+ 
1)) — 5^^^;^ codim as desired. □ 

As an immediate consequence of the above theorem (just as in the case of Sp(2n)), we 
get the following: 
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Corollary 31. Let 1 < m < n and let J^, . . . , J"* G &{m, 2n + 1) be such that 

s 

J][*j,] ^ e H*(OG(m, 2n + 1)). 

Then, lVj=i\.^J^1 7^ ^ H*(Gr(m, 2n + l)). 

4. Comparison of eigencone for Sp(2n) with that of SL(2n) 

Let G be a connected (complex) semisimple group. Choose a maximal compact sub- 
group K olG with Lie algebra t. Then, there is a natural homeomorphism C : B/i^ — 1)_|_, 
where K acts on 5 by the adjoint representation and f)+ is the positive Weyl chamber in 
1^ as in Section 2. The inverse map takes any /i e to the X-conjugacy class of 
ih. 

For a positive integer s, the eigencone is defined as the cone: r(s, i^') := 

s 

{{hi, . . . , /i,) e I 3(A;i, . . . , A;,) e r: J] kj = and C(%) = /i,Vj = 1, . . . , s}. 

Given a standard maximal parabolic subgroup P, let cup denote the corresponding 
fundamental weight, i.e., uop{a^) = 1, if G A \ A(P) and otherwise, where A(P) is 
the set of simple roots for the Levi subgroup L of P containing H. Then, a;p is invariant 
under the Weyl group Wp of P. 

We recall the following theorem from [BeSj] . 

Theorem 32. Let {hi, . . . , hg) G f)^. Then, the following are equivalent: 

(a) {hi,...,hs)er{s,K). 

(h) For every standard maximal parabolic subgroup P inG and every choice of s -tuples 
(wi, . . . , Wg) G {W^y such that 

[A^J ■ . . . ■ [A^J = d[A^] G H*{G/P), for some nonzero d, 

the following inequality holds: 

s 

In fact, assume that (a) is satisfied, i.e., {hi, . . . ,hs) G r{s,K). Then, for every 
standard maximal parabolic subgroup P in G and every choice of s -tuples {wi, . . . , Ws) G 
{W^y such that 

fKj - ■■■■[K.]^o^H*{G/py 

the following inequality holds: 

s 
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Recall that i)^ (respectively, i)^) is the dominant chamber in the Cartan subalgebra 
of Sp(2n) (respectively, S0(2n + 1)) as in Section 2. 

The following theorem is our main result on the comparison of the cigcncone for Sp(2n) 
with that of SL(2n) (and also for S0(2n + 1) with that of SL(2n + 1)). 

Theorem 33. (a) For hi,...,hse l)^, 

(h, ...,h,)e r{s, Sp(2n)) ^ (h, ...,hs)e r(s, SU(2n)). 

(b) Forh,...,hs e 

{hi,...,hs)e r(s, S0(2n + 1)) ^ (/ii, . . . , hs) e r(s, SU{2n + 1)). 

(Observe that by Section 2, \)% C f)^'""' and \)l C f)^^".; 

Proof. Clearly, r(s,Sp(2n)) C V{s^SU{2n)). Conversely, we need to show that if h = 
(/ii,...,/i,) e {\)%Y is such that h G r(s, ^;7(2n)), then h e r(s,Sp(2n)). Take any 
1 <m <n and any . . . , e 6(m, 2n) such that 

[$71] • . . . • [$73] = (i[$e] e i7*(IG(m, 2n)) for some nonzcrod. 

By Corollary 11, 

pii] ■ ... ■ pis] 7^ e if*(Gr(m, 2n)). 
In particular, by Theorem 32 applied to SU (2n) , 

s 

^mC^Vj^hj) < 0, 

where uOm is the m-th fundamental weight of SL(2n) and vu G S2n is the element asso- 
ciated to P as in Subsection 2.1. It is easy to see that the m-th fundamental weight cu^ 
of Sp(2n) is the restriction of ujm to f)*". Moreover, even though the elements Vp G 821% 
and Wii G are, in general, different, we still have 

Applying Theorem 32 for Sp(2n), we get the (a)-part of the theorem. 

The proof for S0(2n-|- 1) is similar. (Apply Corollary 31 instead of Corollary 11.) □ 

5. A BASIC TRANSVERSALITY RESULT 

Let M be a r-dimensional space and = {F}, . . . , F/) an s-tuple of complete flags on 
M. As earlier, let V be a 2n-dimensional vector space equipped with a nondegenerate 
symplectic form, and Q = [G], . . . , G*) an s-tuple of (complete) isotropic flags on V. Let 
/X = (/x^, . . . , /i'^), where /i^ is an ordered sequence with r elements 2n > n{ > ■ ■ ■ > nl > 
0, J G [s]. We flx fjL in this section once and for all. 

Make the deflnition: 

n^{:F, e?) = {0 G Hom(M, V) : 0(Fi) C G{^_^^, a G [r]J G [s]}. 
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(In the next section, we will need to use 'H^{J-',Q) in the case when Gl is an arbitrary 
complete flag on V, not necessarily an isotropic flag.) 

Now assume that {J-',G) is a generic point of F1(M)* x IFl(K)'*, where IFl(y) is the 
full isotropic flag variety of V and F1(M) is the full flag variety of M. Then, 

Theorem 34. The following are equivalent: 

(A) Ti.ix{T, Q) is of the expected dimension 2nr — Yl]=i Ia*''!? where \^^\ :— Y7a=i f^a- 

(B) For any 1 < d < r and subsets B^, . . . , B'^ of [r] each of cardinality d such that 
the product Y[j=i[^BA 7^ ^ H*{GY{d,r)), the inequality X^j=i X^ogBj < 2cin 
holds. 

We record the following corollary. 

Corolleiry 35. Suppose that 2nr — Y^^j=i |a*''| = 0. Then (A) (or (B)) is equivalent to 
the condition 

where is the irreducible representation o/SL(r) as in Section 6.2. 

Proof. The equivalence of the condition in the corollary and condition (B) in Theorem 34 
is a consequence of the Knutson-Tao saturation theorem for SL(n) [KT], together with 

Klyachko's work [K]: these works together characterize the existence of invariants in a 
tensor product of SL(r) representations by a system of inequalities, which is (B) (cf. 
[F4]). □ 

5.1. (A) implies (B) in Theorem 34. This follows from [Bi], Proposition 2.8 (1). The 

idea is that if {B) fails, pick an S* G ^'j=i^Bj{Fi) C Gr((i, M), compute the expected 
dimension of the vector space {0 G 7Y^(JF, Q) : (j){S) = 0}, and find it to be greater than 
the expected dimension of Hfj,{J-',Q). 

5.2. (B) implies (A) in Theorem 34. We proceed by induction on r. To show that 
(B) implies (A), suppose 0° is a general member of T-C^{J-,Q), S — ker(0°) and assume 
S G n^^=ifiB. (Ff ) C Grid, M), for B^ = {b{ < ■ ■ ■ < h>^}. 

We will use ideas of Schofield [S] in the proof. We replace the Ext groups in [S] by 
the cohomology of suitable (2-step) complexes, and replace the long exact sequences in 
cohomology by the snake lemma. There are two other ingredients required (beyond the 
technique of Schofield). 

• A critical dimension count (Proposition 36), for which we need Theorem 10. 

• An idea from [Bi] on genericity of induced structures (we could have used another 
technique of Schofield instead as well) . 

We were unable to use the strategy of [Bi], Section 5 (essentially because we could not 
show that the induced flags on S and V/ S were "mutually generic" in the situation here) . 
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5.3. The set up, and the key inequaUty. Let L = im(0"). Let .A°(Hom(M, V)) — 
Hom(M,y), and 

^^(Hom(M, V)) = e^Li Hom(M, V)/P\ 

where 

= {r G Hom(M, V) : T{Fi) C Gl^_^ a = 1, . . . , r}. 

There is a natural differential (a direct sum of projections) d : A^{llom{M,V)) 
A\Eom{M, V)). Call this (2-term) complex ^*(Hom(M, V j). Clearly, H'^ {A* {Eom{M , V))) 
is the same as 7ifj,{M, V) and x(^*(Hom(M, V"))) is the "expected dimension" of 7Y^(M, V"), 
which is 2nr — ^^^^ l/z-^ j. To make the dependence on {T, Q) and /i precise, we sometimes 
write this complex as A*{M, V, J^, Q, fi). 
Hypothesis (A) is clearly equivalent to 

(C) h\A*{liom{M,V)))^0. 

We have surjections Hom(M, V) A Hom(,S, V) A Hom(,S, V/L). Let 

^°(Hom(5, V")) = Hom(5, V), A%Rom{S, V/L)) = Hom(5, V/L) 

and 

^^(Hom(,S, V)) = e^^^i Hom(,S, V)/r{P^), A\llom(S, V/L)) = Hom(,S, V/ L)/7ror(P^). 
There are natural differentials d : A^ ^ A^ in each case. It is easy to see that 

T{P^) = {(l)eRom{S,V):(P{F^{S),)cGi , ,x = 1, . . . ,d}. 

So, if we let 7^ = /z^-, the (2-term) complex A*(liom{S,V)) is the same as 
A*{S, V, J^{S), g, 7) where J^{S) is the induced s-tuple of flags on S. 

TT o t(P^) = Rom{S, V/L) : 0(F^(>S),) C , + L/L, x^l,...,d}. 

Define numbers 5^ by 

2n - dim(L) - 5^ = 2n - 7^ - dim(G^^_^, n L) 

and hence, ^*(Hom(5, V/L)) = A*{S, V/L, T{S), QiV/L), 5). 

The following critical result will be proved in Section 5.6 by a dimension count. 

Proposition 36. 

h\A*{Yioui{M,V))) < -x{A*{Yioui{S,V/L))). 
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5.4. Proof of (B) implies (A) in Theorem 34 assuming Proposition 36. The 

map of (2-term) complexes A*(liom{M,V)) — > A*(liom{S,V/ L)) is surjective in each 
degree. Therefore, the map 

(30) e : H\A*iliom{M, V))) H\A*{liom{S, V/L))) 

is a surjection. 

Assume Proposition 36. Hence, 

(31) /i'(^*(Hom(M, V))) < -x(^*(Hom(>5, V/L))) < h^{A*{Ylo^{S, V/L))). 

The surjection (30) and the Inequahty (31) imply that 6 is an isomorphism. Thus, 
equality holds in all the inequalities in (31); in particular, hP{A*{^o-m{S,V / L))) — 0. 
The isomorphism 9 factors as 

H\A*{YLoui{M,V))) ^ H\A*{YLo^{S,V))) H\A*{YLo^{S,V/ L))), 
where the map f is induced from the surjective map of complexes 

X(Hom(M, V)) X(Hom(5, V)), 

and is hence surjective. We conclude that f is an isomorphism. 

It follows from [Bi] that the induced flags {T{S), Q) can also be assumed to be suitably 
generic (see Section 5.5). Now, by induction on the dimension of M, assume the validity 
of Theorem 34 with M replaced by S and ji^ replaced by 7-^ . The hypothesis (B) holds 
because of Lemma 37 below. 

Therefore, by conclusion (C) (valid by induction), we find that /i^(.4*(Hom(S', V))) — 
0. Hence ^^(^*(Hom(M, V))) = as desired. This completes the proof of Theorem 34. 

Lemma 37. Let 1 < d' < d. In the above situation, suppose C^, . . . ,C'^ are subsets of 
[d] each of cardinality d' such that [1^=1 [^c^] 7^ e H*{GT{d' ,d)). Then, if = {V^ : 
a e 0}, 

(i) n;=ife]7^0e//*(Gr(d',r)). 

(ii) E3=i Eaec^- = E3=i EbGT^ ^ 2d'n. 

Proof. Since 5" e n*^^fiBj (-^/) C Gr((i, M), and are generic flags on M, the prod- 
uct n3=i[^B^] 7^ e H*{Gr{d,r)). Now using the hypothesis 113=1 [^c^] 7^ e 
H*{Gr{d',d)), and [F3], Proposition 1, we conclude that (i) holds. Because of our as- 
sumption (B), (i) implies the inequality in (ii). The equality in (ii) is trivial. □ 

5.5. Genericity of induced structures. Let U C F1(M)* x IF1(T/)* be a nonempty 
open subset of the space of pair of flags which is generic for our dimension calculations. 
There is a nonempty open subset U C Fl(S')* x IFl(V)'* of points {H, Q') for which (7i, Q') 
is generic for the application of induction on ^*(5', V^, 7i, 7). The projection of U to 
IFl(y)* is a (diagonal) Sp(V^) -invariant open subset of IFl(l^)* which does not depend 
upon the choice of e Gr((i, M). 
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We obtain a point {J-'{S), Q) G Fl(5')'^ x IFl(V)'^ as in the previous section. Since Q is 
generic, it may be assumed to be in the projection of tj to IFl(y)*. Let Ti, G F^^)* be 
close to T{S) (in the complex topology) such that {H, Q) G U. Now find P G F1(M)^ so 
that r{S) = n, T'{M/S) = /{M/S) and S G n|^iQB,(F'^.) (see Lemma 2.4 in [Bi]). 
Clearly (JF', Q) is close to (JF, Q) and therefore is generic in F1(M)^ x Fl(y)* as well (and 
hence belongs to U). Now (p" G H°{A*{M, V, T' , G, ii)) and we replace (JF, g) by {J^', g). 

5.6. Proof of Proposition 36. Since we already know the Euler characteristic of 
A*(llom{M, V)), we want to give a formula for /?°(^*(Hom(M, V))) = n^{T, g). 

If we drop the isotropy condition on the flags ^, then we are in a situation where 
Schofield's original argument can be apphed. We will compute Ti.^{T, ^) by a procedure 
that "essentially" does not see the isotropy condition on the flags g, and hence Schofleld's 
set up generalizes. 

To calculate the dimension of 7i^(jF, g), we can flber the universal parameter space of 
triples {(j),J-',g) with generic in 7if^{J-', Q), over the parameter space of pairs (im(0), g) 
so that im(0) is in its generic Schubert state with respect to the flag g. The dimension 
of the choices of such pairs is the expected one, thanks to our main transversality result 
Theorem 10. 

5.7. Calculation of x{A*(liom{S, V/L))). First, as in Subsection 5.3, tt o t{P^) is the 
same as 

{6 G Rom{S, V/L) : 9{F\S),) C + L/L, x = 1, . . . , d}, 

which has dimension X)f=i(2n-7^ -dim(LnG'^' J). Therefore, x(^*(Hom(,S, V/L))) 
equals 

s d 

d(2n -r + d)- 5^(ci(2n - r + d) - ^(2n - 7^ - dim(L n G^^^))) 

j=l x=l 



s d 

(32) = d{2n - r + d) + ^ J](r - d - 7^ - dim(L n Gl_^)) 

j = l x = l 

5.8. The dimension of 'H^{J^,Q): Assume that 

L G n'j^^QAj{Gi) C Gr(r - d,V). 

Let Ua{V, s) C IFl(l^)* be the open subset consisting of isotropic flags El, . . . , such 
that the intersection 0^=1^1 (-E'f) inside Gr(r — d, V) is proper. By Theorem 10, this is 
nonempty. Let U denote the parameter space 

{(^, g, 4>,s,L):Te Fl(M)^ g G Ua{v, s),Le n'^^.n^^iGi), S G n^UQB.(i^/), 

G n^{J^, g),S^ ker(0), L = im(0)}. 
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Now, the dimension of 7Y^(JF, Q) equals the dimension of a generic fiber of W — > FL(M)* x 
Ua{V, s). We will therefore need to give an upper bound for the dimension of irreducible 
components of U. 

Clearly, U maps to the parameter space 

V = {{g, (f>,S,L):g e Ua{V, s), L e n'^^^QA^iGi), S e Gr{d, M), cf) e Hom(M, V), 
S — ker(0), L — im(0)}. 

5.8.1. The dimension ofV. Clearly V fibers over the space 

W = {{L,g) : g e UA{V,s),Le r\'j=,QA.{Gi)} 

with irreducible fibers of dimension d{r—d) + {r—d)'^ (the first term is the dimension of the 
space of choices of S and the second term is the dimension of the space of isomorphisms 

M/S^L). 

Now the fiber dimension of W ^ Ua{V, s) is ((r — d){2n — r + d) — X]^=i codim(Q^3)). 
Therefore, the dimension of any irreducible component of V is no more than 

s 

dim(IFl(l^)*) + ((r - d){2n - r + d) -J^ codim(l]^.)) + d{r - d) + {r - df. 
By simplifying the above expression, we find 

s 

(33) dim V <2nr + {r-d- 2n)d + dim(IFl(\/)") - ^ codim(n^,). 

5.8.2. Codimension offlAi- 

Lemma 38. dim(Q^j) < '^a^sA'^''^ ~ f^a ~ dim(L fl j)), and hence 
codim(Q^, ) > if^i + dim(L n Gl_^ ) - (r - d)) . 

Proof. Let = {oi < ■■■ < flr-d}- Therefore, dim(r2^j) equals Ylb=i(^b Let 
[r] \ = {xi < ■ ■ ■ < Xr-d}- It suffices therefore to show that, for any 1 < b < r — d, 



a,-h<2n- i^i^ - dim(L n , ), 



I.e., 



(34) a,<b + 2n- i^i^ - dim(L n ) . 

Set dim(L fl ^ ) = 6 + y, where y > (since LnG^l j contains 4>{F^ ) which is 
6-dimensional) . 

Hence, to prove Equation (34), we need to show that L fl G-' j is at least b- 

dimensional, which is now immediate because L n G-* ^ is (6 + y)-dimensional. □ 

2n fJ'x^ 
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5.8.3. The fiber dimension of U — >■ V. We need to do the following basic calculation. 
Let S (Z M he a d-dimensional subspace, L a subspace of V of dimension r — d and 
B e S{d,r). 

Lemma 39. Let G, he a fixed flag on V , and (p : M ^ V a map with kernel S and 
image L e D,a{G,). The dimension of the space of flags F, e F1(M) so that S e Qs(F,) 
and (f){Fa) C G'2n-/ia for a ^ 1, . . . ,r, equals 

dim(Fl(M)) - (codim(l]B) + ^(r -d- dim(L n Gan-^J))- 

Proof. There are two sets of conditions imposed on F,. The first one is that S e 

the second is that C G2n-fia H L for a ^ S. These conditions are independent. □ 

Therefore the fiber dimension of — > V equals 

s 

(35) dim(Fl(M)*) - V(codim(OB, ) + V (r - d - dim(L n G^ _ , ))). 

i=l a^Bi 

5.9. Conclusion of the proof of Proposition 36. The dimension of any irreducible 
component of U is less than or equal to the sum of the fiber dimension of — > V (given 
by (35)) and the right hand side of Inequality (33). This sum equals 

s 

dim(Fl(M)^) - y (codim(nB^) + V (r - d - dim(L n & _ , ))) 

(36) "^^^ 

+ 2nr + (r - d - 2n)d + dim(IFl(y)^) - ^ codim(Q^i). 

The dimension of l-iy^iT , ^), which equals the dimension of the generic fiber of the map 
U. — > F1(M)* X UjSy,^), is therefore less than or equal to the integer (36) minus the 
dimension of F1(M)* x C/a(\/,s). Therefore, using Lemma 38, 

s 

di^{H,{T, g)) < - ^(codim(05.) +Y,{r-d- dim(L n &^^_^^))) 

s 

+ 2nr + d(r-d-2n)-J2 J2U + ^^(^ ^ ^L-^ " " ^)) 

s s 

— 2nr + d{r — d — 2n) — codim(r2Bj) — /x;^. 
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Now, h\A*{M,V)) equals the dimension oin^,{T,Q) minus the Euler characteristic 
which is 2nr — Yl]=i Z]I=i A*o- This yields that h}-{A*{M, V)) is less than or equal to 

s s 

(37) d(r-d-2n)-J2 codim(Qsi ) + J^^i , 

i=l i=l a<=Bj 

which can be written in an illuminating (and expected) form 

s 

(dimGr(d, r)-J2 codim(ns, )) + ( J] ~ 
Comparing Equation (32) with Equation (37), it suffices to show (for each j) 

d d 
x=l a&Bi x=l 

which rearranges to 

d d 
x=l x=l 

But, this is clear because L fl ^ contains the — x dimensional subspace (j){F^j). 
This completes the proof of Proposition 36 and hence Theorem 34 is proved. □ 

6. Tensor product decomposition for SL(r) versus Sp(2n) 
Let G, B be as in the beginning of Section 2. 

6.1. Line bundles on G/B. For a character x : S — > C*, let be the corresponding 

G-equivariant (homogeneous) line bundle on G/B associated to the principal S-bundle 
G G/B via the character x^^- Recall that its total space is the set of all pairs 
{{g,c) : g & G,c E C} modulo the equivalence 

{9,c) ^ {gb,x{b)c). 

If X is a dominant weight, then 



H"iG/B,C^) = V^ 



where (as in Section 2) is the irreducible representation of G with highest weight x- 
If / : G' — > G is a map of algebraic groups so that f{B') C B, where B' and B are 
Borel subgroups of G' and G. Then, / induces a map / : G'/B' G/B. It can be 
checked that f*jO,^ — J0,^of, where x o / is the composition of x with f\B' B' ^ B. 

Specific identification in the SL(r) case: Let B be the standard Borel subgroup 
of SL(r) consisting of upper triangular matrices as in Subsection 2.1. Then, the full 
flag variety Fl(r) = SL{r)/B can be identified with the space of full flags F, on a r- 
dimensional vector space M. As such it receives natural line bundles = det(Fa)* 
(1 < a < r). This line bundle is isomorphic with the line bundle jCi,^^, where cua is the 
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character of B which takes the diagonal matrix (ti, . . . , t,.) to the product YYi=i ^i- Recall 
that cua is the a*^ fundamental weight. 

6.2. Representations of Gl(r). Irreducible polynomial representations of the general 
linear group Gl(r) correspond to the sequences of integers /i — {/ii > • • • > /^r > 0) (also 
called Young diagrams or partitions). Given a sequence of integers /i — {/ii > /i2 > • • • > 
A*r- > 0), form the line bundle 

r 

a=l 

where {fir+i = 0). Then, H^(F\{r), C^) = V*, where is the irreducible representation 
of Gl(r) with highest weight /i. To any such n, we associate the character /i of 5 by 

r 

fl{t) = JJ^ , where t = diag(ii, .. .,tr). 

i=l 

Then, clearly, £^ ~ 

6.3. A Basic Calculation. As earlier in Subsection 2.2, let V — C^" be equipped with 
the symplectic form ( , ) and M a vector space of dimension r. 

We will now make a basic determinantal calculation (which appeared originally in a 
weaker form in [B2], Lemma 4.5). Let P := Hom(M, V). Fix an integer m < 2n. For a 
full flag F, on M, a full flag G, on V, and a partition = (m > //i > • • • > //^ ^ 0), let 
PmilJ', F,, G,) denote the subspacc 

{0 e Hom(M, V) : 0(F„) C G^.^^, Va e [r]}. 

Consider the vector bundle Tmin) (respectively, Pm{fi)) on F1(M) x F1(K) whose fiber 
over (F., G.) is T^{fi, F., G,) := Hom(M, V)/Pm{l^, F., G.) (respectively, P^(/x, F., G.)). 
We now calculate the determinant line bundles of PmilA and T^d^). We begin the 
computation by setting (for 1 < 6 < r) 

= {0 e Hom(M, V) : (f){F,) C G^_^„, a = 1, . . . , 6}. 

There exist exact sequences for < 6 < r — 1: 

^ P4+^ ^P^^ Hom(F,+i/F,, y/G^_^,^J ^ 0, 

where P^ := Hom(M, l^). This permits us to write (£„ := det{Fa)\ICb det(Gb)*) 

r r 

detTM = ((n>C^"~'^''+0 ® (detM*)^-— ) K ((detl/)'^ ® H^— mJ' 

a=l a=l 

where /x^+i 0. (Here we have used the fact that for vector spaces Vi, V2, det(Hom(V^i, V2)) — 
((detyi)*)'^™^2 ^ ((jgt y^^dimVi ^ Lg|. ;^ ^jjg partition conjugate to the partition A' := 

(m — > • • • > m — /Xi > 0). We call A to be obtained from /i by m-flip. Then, we may 
rewrite the above formula as 

(38) det TM = (det M*)^"-"^) m (^{det Vy ® a) ■ 
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6.4. The theta section. Let M and V be as in Section 6.3. We take m = 2n. Let 
{lJ'-'}i<j<s be partitions: 

2n>n{>--->ni>0 

satisfying 

s 

ln^l = 2nr, 

where := Y.l=i 1^1 

Consider two GL(M) x Gl(V^)-equivariant bundles V and Q on F1(M)^ x F\{VY (with 
the diagonal actions of G1(M) and Gl(V^) on F1(M)* and Fl(\^)* respectively). The fiber 
of V over (JF, ^) is just the constant vector space Hom(Af, V) (with the natural action 
of the group GL(M) x Gl{V)), where = {F}, F/) is an s-tuple of fuU flags on M 
and similarly for Q. The fiber of Q over (J^, ^) is the direct sum 

©•=iT(/x^F^Gi), 

where T(/x^ F.^', G^J := T2n{fi^ , F^, Gi). Consider the canonical GL(M) x Gl(\/)-equivariant 
map S : V ^ Q between vector bundles (of the same rank) on F1(M)* x Fl(V)'* in- 
duced via the quotient maps. Thus, taking its determinant, we find a GL(M) x G1(F)- 

equivariant global section G det(Q) ® (dctP)^^ such that vanishes at {J^^G) if and 
only if 1-L^{T^Q) ^ 0, where Ti^^T^Q) is as defined in the beginning of Section 5 (ex- 
tended for any, not necessarily isotropic, complete flags Q = {G^, . . . , G^)). By Equation 
(38), 

det(Q) = C^im-'-mC^sM {det{Vy ® ^0 K ■ ■ ■ K (det(V)" ® Lx^). 

Here A* is obtained from by 2n-flip. 

Fixing a point G Fl(M)'^, on restriction, we get (as in [B2]) a SL(y)-invariant section 
0jr in 

H%Fl{vy, K • • • K Cx^f^^^l 
Restricting this to IFl(y)* we find a section in 

where IFl(y) is the space of (full) isotropic flags on ^ , Xq is the weight of Sp(2n) obtained 
by the restriction of A' to the Cartan of Sp(2n) and is the irreducible representation 
of Sp(2n) with highest weight u. (Observe that for a dominant weight A of SL(2n), its 
restriction Ac is dominant for Sp(2n).) 

We can therefore record the following consequence of Corollary 35. 

Corollary 40. Let {fJ'-^}i<j<s be partitions: {2n > /^i > • • • > //^ > 0) such that 
I A*'' I — 2^^- Assume further that 
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Then, 



where := and is obtained from fi^ by 2n-flip. 

We now come to the main theorem of this section. 

Theorem 41. Let Vxi, . . . ,Vxs be irreducible representations o/ SL(2n) (with highest 
weights A^,...,A*' respectively) such that their tensor product has a nonzero SL(2n)- 
invariant. Then, the tensor product of the representations ofSp{2n) with highest weights 
A^, . . . , A^ has a nonzero Sp{2n) -invariant, where A^ is the restriction of X^ to the 
Cartan f)*^ o/Sp(2n). 

Proof. We would hke to find some r and partitions //^, . . . , such that 

2n>iJ,{>--->ni>0 

with "^j^i |a*-' | = 2nr, and such that A'' is obtained from /i^ by 2n-flip. 
To achieve this, express A'' as sequences of integers (with r large) 

r > A{ > • • • > Ai > 

so that (since the tensor product of the corresponding representations of SL(2n) has a 
nonzero SL(2n)-invariant) 

s 2n 

A^ = 2nb' for some integer 6' > 0, 

j=l a=l 

i.e., 

s 2n 

J]J](^-Ai)=2nr + 2n6, 

j=l a=l 

where b :— r{s — 1) — b'. Taking r > -^X{ (for every j), we can assume that b > 0. 
Rewrite the above equahty as X]^=i Z]a=i i'r' + b— {b-\- Xi)) — 2n{r + b) . So, we replace r 
by r + 6 and A^ by 6 + A^ to assume b — 0. Let //-^ be obtained from A^ by r-flip. Since, 
by assumption, 

we obtain (using ordinary duality) 

where X^' — {r — A^„, . . . , r — X{). 

Now using Grassmann duality (see Section 6.5), 

Now, apply Corollary 40. (Observe that 2n-flip of is the partition A^ — A2„ > • • • > 
"^in-i ~ ^2n ^ 0-) This completes the proof of the theorem. □ 
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6.5. Grassmann duality. Let r and k be positive integers and let m = r + k. Given a 
Young diagram n — {k > /ii > ■ ■ ■ > /ir > 0), one obtains naturally a cohomology class 

Lo^ of Gr(r, m), which is the cycle class of flA{fj,) with A{fj,) = {m — r + a — /ia|a G [r]}. 
It is known that if //^, . . . , fi^ are such Young diagrams with l/^-' l = kr, then the 

/ \ SL(r) 

dimension of (V^i ® ■ ■ ■ ® V"^sj equals the coefficient of the cycle class of a point in 
the cup product 11^=1 ^^j^ ^ H'^^'^ {Gi {r,m)) (see, e.g., [Fi], Chapter 9). 

Clearly, under the natural duality map Gr(r, C™) = Gr(/c, (C™)*), the class a;^ goes 
to (Vjj,, where fl is the conjugate diagram to (see, e.g., [Fi], Exercise 20, page 152). We 
therefore obtain the following consequence, which we call the Grassmann duality. 

Lemma 42. Suppose J2^j=i I A*'' I — Then, 

dim(V^i (g)---(g) V^sf''^''^ = dim(\/^i ® ■ ■ • ® V-^sf''^''\ 

We clearly have another "ordinary duality". Let be the dual partition to fi^ (so 
that = {k — i^l, . . . , k — /^i)). Then, clearly, V^j is the SL(r)-dual of V^j and hence 

dim(T/^i (8) • • • ® V^.f^^"^^ = dim(Ki K^^^^'^- 
7. Saturation for Sp(2n) and S0(2n + 1) 

Let X(i7*")+ be the set of dominant characters of H'-^ (for the group Sp(2n)) and 
similarly let X{H^)^ be the set of dominant characters of (for the group S0(2n + 
1)). We have the following saturation theorems for the groups Sp(2n) and S0(2n + 1) 
respectively. 

Theorem 43. Given v^, . . . & X{H'-^)+, the following are equivalent: 

(1) For some positive integer N, the tensor product of Sp{2n) -representations with 
highest weights Nu^, . . . , Nw'^ has a nonzero Sp{2n) -invariant. 

(2) The tensor product of representations with highest weights 2i/^, . . . , 2i/* has a 
nonzero Sp(2n) -invariant. 

Proof. For any semisimple algebra Q with Cartan subalgebra f), we identify k, : i) — ^ f)* 
via the normalized invariant bilinear form ( , ) on f)*, normalized so that {6, 6) ~ 2 for 
the highest root 6. Under this identification, f)+ — > D and, moreover, Xi i— > 2uji/ (ccj, ctj). 
For any s > 1 and G as in the beginning of Section 2, define 

f (s, G) = {(A\ . . . , A') e {X{H)^y : for some > 1, 

Then, under the identification induced by k (and still denoted by) k : 1^^ — > D^, we have 
(cf., e.g., [Sj, Theorem 7.6]) 

K-\t{s, G)) = r(s, K) n K-\{x{H)+y), 

where the eigencone r(s, K) is defined in Section 4. 
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Recall from Subsection 2.2 the inclusion i : ^ [^Mn^i g^j^^ hence we get the dual 
(surjective) map i* : ^l)'^^"-!^ -» (I)*")*. We have the following commutative diagram: 

(*) 2kc i^a 

where kc (respectively, ka) denotes k for Sp(2n) (respectively, SL(2n)). 

We only need to prove (1) ^ (2). By assumption, v := {u^, • ■ ■ , z/*) G T{s, Sp(2n)) 
and hence k^^u G r(s, Sp(2n)). But, clearly, r(s, Sp(2n)) C T{s, SU{2n)) and hence 
KAinc\i^) G f(s, SL(2n)). (Observe that KAi Kc\X{H^)+) C Q^^n-i nX{H^^"-^)+, 
since nciPj) — Pj for 1 < j < n and nc{Pn) — \ Pn, where Q'^^""^ is the root lattice of 
SL(2n).) Moreover, since kai Kq^{i/^) G (5"^^""% by the saturation theorem of Knutson- 
Tao [KT], 

where :~ kai Kq^{i'^) and Vxj is the irreducible representation of SL(2n) with highest 
weight A-' . 

Applying Theorem 41, we get 

/ \ Sp(2n) 

i^M^i^ ® • • • ® M^sj ^ 0, 

where A;^ := Aj ^ and M^j is the irreducible representation of Sp(2n) with highest weight 

A;^. Moreover, by the commutativity of the diagram (*), we get = 2vK This proves 
the theorem. □ 

By using [KS, Section 4] and the above theorem, we get the corresponding result for 
the odd orthogonal groups. 

Theorem 44. Given v^, . . . ,1/^ G X{H^) + , the following are equivalent: 

(1) For some positive integer N, the tensor product of SO {2n + 1) -representations 
with highest weights Nu^, . . . , Nu^ has a nonzero S0(2n + l)-invariant. 

(2) The tensor product of representations with highest weights 21/^ , ... ,21/^ has a 
nonzero S0(2n + l)-invariant. 

Remcirk 45. (a) The general saturation result proved by Kapovich-Millson [KM] spe- 
cialized to the case of Sp(2n) requires saturation factor of A: as opposed to our satura- 
tion factor of 2 proved in Theorem 43. Our Theorem 43 is optimal for general weights 
i/^...,z/^ G X{H^)+. 

(b) We can prove an analogue of Theorem 34 for S0(2n+ 1) by a similar method and 
use it to give a proof of Theorem 44 analogous to that of Theorem 43. 
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As an immediate consequence of Theorem 44, we get tlie following special case of the 
general saturation result due to Kapovich-Millson [KM] . 

Corollary 46. Let i/^, . . . , z/^ be dominant characters for Spin(2n + 1) (whose sum is 
not necessarily in the root lattice). Then, the following are equivalent: 

(1) For some positive integer N, the tensor product of representations o/Spin(2n + l) 

with highest weights Nu^, . . . , Nw^ has a nonzero Spin(2n + l)-invariant. 

(2) The tensor product of representations with highest weights iu^ , . . . , ii/^ has a 
nonzero Spin(2n + l)-invariant. 

We would like to conjecture the following generalization of Theorems 10, 30 and 41. 
Let G be a connected simply- connected, semisimple complex algebraic group and let 
cr be a diagram automorphism of G with fixed subgroup K. 

Conjecture 47. (a) For any standard parabolic subgroup P and any Bruhat cells A^^ , ■ ■ ■ , A^ 
in G/P, there exist elements ki,--- ,ks G K such that the intersection f]i=ikiA^. is 
proper. 

(b) Let Vxi, ■ ■ ■ , Vxs be irreducible representations ofG (with highest weights A"^, • • • , A* 
respectively) such that their tensor product has a nonzero G-invariant. Then, the tensor 
product of irreducible K -modules with highest weights A]^, ■ ■ ■ , A|^ has a nonzero K- 
invariant, where A^ is the restriction of to the Cartan subalgebra \}k '■— h'^ of K. 

( Observe that A^ is dominant for K for any dominant weight X of G with respect to 
the Borel subgroup := B'^ of K.) 

8. Horn's problem for symplectic groups 

In this section we prove an analogue of Horn's problem for symplectic groups. 
Fix r <n and let e 6(r, 2n) be such that 

s 

(39) ^codim($j, ) = dimIG(r, 2n). 

Let I3j be the order preserving bijection of P U with [2r] and let be the subset 
I3{P) of [2r] (of cardinality r). For any a G [r] and j G [s], let A^ := > P\ and 
:= 2n — 2r — A{, where P = {i{ < ■ ■ ■ < ii}. Recall the definition of the deformed 
product ©0 in the cohomology H*{G/P) from [BK, Definition 18]. 

Theorem 48. Under the assumption (39), the following conditions (a) and {(3) are 
equivalent: 

(«) n;=i[^/^] 7^ G //*(IG(r,2n),0o). 
(P) (A) dimIG(r, 2r) = ^osym^P). 

m UUi[^ii]y^^^H*{IG{r,2r)). 
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Remark 49. (1) Assume that the condition is satisfied. Then, the condition {(32) 
is equivalent to the following condition. 

For any 1 < d < r and subsets B^, . . . , B'^ of [r] each of cardinality d such that the 
product Y[j=i[^Bj] 7^ e H*{GY{d,r)), the following inequality holds 

s 

J]^/.i<2d(n-r). 

This follows from Corollary 35 applied to a space M of dimension r and V — M^/M 
of dimension 2n — 2r and Identity (41)- In fact, we only need the equivalence of the 
assertion in Corollary 35 and the {B)-part of Theorem 34, which is due to Klyachko and 
Knutson-Tao (of., the proof of Corollary 35). 

(2) The condition (P2) in the above theorem is recursive by the work of Purbhoo and 
Sottile [PS]. The Purbhoo-Sottile recursion says that {(32) is equivalent to a system of 
inequalities indexed by nonvanishing s-fold intersections of Schubert cycle classes in the 
Grassmannians Gr(ci, r) where < d <r. 

8.1. Proof of Theorem 48. We first observe that, under the assumption (39), the 
following two conditions are equivalent. 

(71) n;=i[*/^] ^ e i/*(IG(r,2n),0o). 

(72) nj=i[*/^] ^ e H*{lG{r,2n)) and the condition (pi) is satisfied. 
To see this we use [BK, Theorem 15] and the following lemma. 

Lemma 50. For I G &{r, 2n), Xwii^r) — codim($/) +codim($/^), where Xwj ^■^ defined 

just before Lemma 22. 

Proof. Consider the exact sequence (8). Let Lm = GL(M) x Sp{M-^ /M) be the Levi 
subgroup of the stabilizer Pm of M in Sp(2n). The connected center C* of Lm (which 
is the center of GL(M)) acts as t t-^ t~^ (multiplication by t'^) on Rom{M,M^/M) 
and by t on sym^(M)*. We conclude the proof by using Proposition 16 and the 

formula for Xwi given at the end of page 192 in [BK]. □ 

Let V he a vector space of dimension 2n with a nondegenerate symplectic form and 
let M C be an isotropic subspace of dimension r. Let X = lG(r, 2n) and assume the 
condition (/3i). Let E. = {E}, . . . , E^) be an s-tuplc of isotropic flags on V such that for 
all J e [s], M e ^jj{Ei). Consider the exact sequence (as in Lemma 13): 

^ Hom(M, M^/M) T{X)m ^ sym^ M* 0. 

Let us abbreviate 

= Hom(M, M^/M), B° = T(X)m, C° = sym^ M*. 

For j = 1, . . . , s, let 

P^ := Hom(M, M^/M) n (T($p(£;f))M), := T{^i,{Ei))M, = (/>(r($,.(£;^))M). 
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Define 



There are natural difi^erentials 



A'^A\ B'^B\ C'^C\ 



and an exact sequence of two-term complexes 



(40) 







A 



B 



C* 



0. 



Our assumption (39) implies that x{^*) = 0) ^-iid the assumption {f3i) (in view of 
Proposition 16 (3)) implies that x(C*) = 0. Therefore, from the above exact sequence, 
X{A*) = as well. Thus, by Proposition 16 (l)-(2). 



Moreover, h^{B*) = if and only if hP{A*) — and hP{C*) — from the above exact 
sequence (40). 
We now prove: 

(a) =^ {(3): Let E, = [E], . . . , El) be a generic s-tuple of isotropic flags on V such that for 
all j e [s], M e ^jj{Ei). This can be achieved by the virtue of (a) and by simultaneously 
translating each Ei by the same element of Sp(2n). Since E. is generic, fl = flj^i^ij {El) 
is a transverse intersection at M and hence h}{B*) — 0. Thus, hP{B*) = as well and 
we get the following: 

• h^{A*) = 0. Hence, by Proposition 16 (l)-(2). Identity (41) and Corollary 35 
apphed to V ^ M-^/M, condition (/^a) holds. 

• hP{C*) — which implies condition (/^s) (using Lemma 18, Proposition 16 (3) 
and the condition (/3i)). 

{13) =^ (a): Assume {f3). Find a s-tuple S, = [E^, . . . ,El) of isotropic flags on V such 
that M lies in the intersection Q = n*^]^$/j (-Ef ). The parameter space of such S, is 
irreducible (Lemma 17); pick a generic such S,. It follows that the induced flags on M 
and M-^/M are generic. Therefore, our assumptions (/9i), {P2): Identity (41), Corollary 35 
applied to V = M^/M and Proposition 16 (l)-(2) imply that h^{A*) = 0. Moreover, 
our assumption (/^s), Proposition 16 (3) and Lemma 18 imply that h^{C*) = 0. Hence, 
h^{B*) = and thus h^{B*) = 0. Therefore, ^2 is a transverse intersection at M, and 
hence (a) holds by the equivalence of (71) and (72). This proves the theorem. □ 



s 



(41) 




9. Horn' 



S PROBLEM FOR ODD ORTHOGONAL GROUPS 



In this section we extend the results from the last section to the odd orthogonal groups. 
The proofs are similar, so we will be brief. Refer to the Section 2.3 for various notation. 
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Fix r <n and let J\ . . . , J* G &{r, 2n + 1) be such that 

s 

(42) ^codim(*ji) = dimOG(r, 2n + l). 

Let Qj be the largest integer < n in P U J'^ and set bj := 2n + 2 — aj. Let Pj be 
the order preserving bijection of {J^ U J'-') \ {aj,bj} with [2r - 2] and let be the 
subset Pj{J^ \ {aj,bj}) of [2r — 2] (of cardinality r — 1). For any a e [r] and j e [s], 
let := \ii > and /4 := 2n + 1 - 2r - A{, where ^ {i{ < ■ ■ ■ < ii} and 
J^' := [2n+l]\{J^ UJ'^). 

Theorem 51. Under the assumption (42), the following conditions (a) and {(5) are 
equivalent. 

(«) Y[]=A^jA ^ G i7*(OG(r,2n + l),0o). 

(/?) (/3i) dimOG(r, 2r) = "Yl^^j^i^o !\^(^J^\ where OG(r, 2r) is the orthogonal Grass- 
mannian of isotropic r- dimensional subspaces in the 2r -dimensional space 
with a symmetric nondegenerate form. 

iPs) UUA^ji] ^ G E\\G{r - 1, 2r - 2)). 
Similar to Remark 49, we have the following: 

Remark 52. Assume that the condition (/3i) is satisfied. Then, the condition {132) is 
equivalent to the following condition. 

For any 1 < d < r and subsets B^, . . . of [r] each of cardinality d such that the 
product Y[j=i[^Bi] 7^ G H*{Gr{d,r)), the following inequality holds 

s 

J2J2l'a<d{2n + l-2r). 

i=l a&Bi 

Before we can complete the proof of the theorem, we need an interplay between the 
following three homogeneous spaces. 

9.1. Fundamental triple of classical homogenous spaces. As is well known, the 
following three homogenous spaces have 'essentially' equivalent intersection theory. 

(1) The isotropic Grassmannian IG(r — 1, 2r — 2). 

(2) The orthogonal Grassmannian OG(r — 1, 2r — 1). 

(3) The orthogonal Grassmannians OG^(r, 2r), where OG^(r, 2r) are the two com- 
ponents of the orthogonal Grassmannian OG(r, 2r). 

The relation between (1) and (2) is discussed in the appendix. The spaces (2) and (3) 
are actually homeomorphic. Let y be a 2r-dimensional vector space with a symmetric 
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nondegenerate bilinear form. Fix a subspace T of V of dimension 2r — 1, such that the 
quadratic form is nondegenerate on T. Then, there are homeomorphisms 

ip^ : OG^(r, V) ^ OG(r - 1, T), 

which send to iV fl T. Notice that can not be a subspace of T (because T does not 
have isotropic subspaces of dimension r). 

Fix one component and denote it by OG'''(r, 2r). Then, as in [BL, §3.5], the Schubert 
cells in 00"*" (r, 2r) are parameterized by the subsets / C [2r] of cardinality r satisfying: 

• / n / = 0, where 7 := {2r + 1 - a\a G /}. 

• The number of elements in / that are less than or equal to r has the same parity 
as r. 

One easily sees that the above set is in bijection with 6(r — 1, 2r — 2), the indexing set 
for the Schubert cells in IG(r — l,2r — 2). This bijection takes / to /?(/ \ {r, r + 1}), 
where (3 is the order preserving bijection [2r] \ {r, r + 1} ^ [2r — 2]. By Theorem 55 
and the above homeomorphism ^p^ , this bijection preserves non-zeroness of intersection 
numbers. 

9.2. Proof of Theorem 51. Under the assumption (42), the following two conditions 
are equivalent. 

(71) n;=i[^J^] ^ e i/*(0G(r,2n+ l),0o). 

(72) n5=i[^J^] 7^ e H*{OG{r, 2n + 1)) and the condition (/3i) is satisfied. 

A straightforward reworking of the proof of Theorem 48 yields Theorem 51 with con- 
dition (Ps) replaced by the following: 

(/^g) For generic complete flags F,^,...,F/ on a r-dimensional vector space M, the 
vector space 

(43) {7 e A'M* I 7(i^i,i^^) = 0, a e [r],j e [s]} 
is of the expected dimension 

s 

r(r -l)/2-^ CO a2(J^), 
i=i 

where P := {i{ < • • • < i^} and ti := \J'^ > ill- 
In the following lemma, we will consider only those isotropic flags F, on V such 
that Fr G OG^(r, 2r). We have the following analogue of Proposition 16 (3) for the 
Grassmannian OG"'"(r, 2r). 

Lemma 53. The tangent space at a point P to the shifted Schubert cell in OG'''(r, 2r) 
parameterized by I — {ii < ■ • ■ < ir} and corresponding to an isotropic flag F, on V is 
given by 

(44) {7 G A^P* I ^(F(P),,F{P)J = 0,Va G [r]}. 
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where ta := |/ > ia\ and (as earlier) F{P), is the filtration on P induced from that of 
F,. 

Let Sr be the permutation in the symmetric group S'2r which transposes r and r + 1. 
Then, observe that in the above lemma, the space (44) corresponding to / is the same 
as that for Sr{I). 

Using an obvious analogue of Lemma 18 (with IG(r, 2r) replaced by OG~^{r, 2r)) and 
the relation between structure constants in OG^(r, 2r) and IG(r — 1, 2r — 2) as given in 
Subsection 9.1, we conclude that the conditions (/Js) and (/Jg) are the same. This proves 
Theorem 51. □ 



10. Appendix. Relation between intersection theory of homogenous 

SPACES FOR Sp(2n) AND S0(2n + 1) 

For completeness, we include a prof of the following well-known Theorem 55 (cf., 
e.g., [BS, pages 2674-75]). As in Section 2, there is a canonical Weyl group equivariant 
identification between the Cartan subalgebras 1^*^ and f)^ (and also the Weyl groups W'-^ 
and W^) of Sp(2n) and S0(2n + 1) respectively. We will make these identifications and 
denote them by [) and W. Let Af (respectively, Af) be the BGG operators for Sp(2n) 
(respectively, S0(2n + 1)) acting on S{i)*) = C[ei, . . . , e„] corresponding to the simple 
reflections s,, where {ei, . . . , e„} is the basis of f)* as in [Bo, Planche II and III]. 

Lemma 54. 

Af^Af, l<i<n 




For any w eW with reduced decomposition w — ■ ■ ■ Si^, deflne A^ — Af^ ■ ■ ■ Af^ and 

similarly define A^. (They do not depend upon the choice of the reduced decomposition.) 
Consider the isomorphism 

: H* (S0{2n + l)/B^, c) ^ H* (Sp{2n)/B^, c) , 

induced from the Borel isomorphism via the identity map 

C[6i, . . . ,6n] £ C[ei, ...,£„] 

where = /*" is the ideal generated by the positive degree l^-invariants in C[ei, . . . , e„]. 
Let {[A^{C)]}yj^w C. H*{Sp{2n)/B'^) be the Schubert basis and similarly for {[A^(i?)]}^gvy- 
Let e C[ei, . . . , e„] be a representative of [Ae(C)] and similarly pf for [Ae(-B)]. Then, 
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by [BGG], we can take 




l<i<j<n 




and 



W 




l<i<j<n 




W 



Thus, 




By loc cit., := A^.ip"^ represents the class [A^(C)] and similarly := A^_ip^ 
represents the class [A^„(S)]. Hence 



where (as in Section 3) ij.{w) represents the number of times the simple reflection s, 
appears in any reduced decomposition of w. Thus, we have the following: 

Theorem 55. The algebra isomorphism satisfies: 
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